Oct 30 20:59:37.317162 ip-10-0-140-179 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 20:59:37.720429 ip-10-0-140-179 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:59:37.720429 ip-10-0-140-179 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 20:59:37.720429 ip-10-0-140-179 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:59:37.720429 ip-10-0-140-179 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:59:37.725340 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725236    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.725340 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725332    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.725340 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725337    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.725340 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725341    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.725340 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725345    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.725340 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725349    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725353    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725357    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725361    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725365    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725369    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725373    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725376    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725379    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725383    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725388    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725392    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725396    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725400    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725403    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725407    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725411    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725422    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725426    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.725682 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725433    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725439    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725443    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725447    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725451    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725456    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725459    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725463    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725467    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725471    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725475    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725479    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725483    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725488    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725492    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725497    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725501    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725506    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725510    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725514    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.726461 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725518    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725522    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725528    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725532    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725537    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725542    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725546    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725550    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725554    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725558    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725562    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725566    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725570    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725574    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725579    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725582    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725587    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725591    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725596    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725600    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.727292 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725604    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725608    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725613    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725617    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725622    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725626    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725631    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725636    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725642    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725647    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725652    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725660    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725667    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725673    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725678    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725683    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725688    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725692    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725696    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.728129 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725700    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725704    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725708    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725713    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725717    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725722    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725726    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725730    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725735    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725739    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725743    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725747    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725751    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725756    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725760    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725765    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725769    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725773    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725780    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725784    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.728938 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725789    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725793    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.725798    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727172    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727183    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727188    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727192    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727196    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727201    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727205    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727209    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727213    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727218    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727223    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727227    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727231    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727235    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727240    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727244    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727249    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.729557 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727253    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727258    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727262    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727267    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727271    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727275    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727280    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727284    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727288    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727293    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727297    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727300    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727304    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727308    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727312    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727318    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727323    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727327    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727331    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727336    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.730145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727340    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727344    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727349    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727353    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727357    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727362    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727366    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727370    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727374    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727381    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727385    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727389    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727394    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727398    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727402    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727406    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727410    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727415    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727419    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.731025 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727423    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727427    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727431    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727436    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727440    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727444    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727449    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727453    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727457    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727462    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727469    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727476    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727481    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727486    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727491    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727495    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727499    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727503    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727508    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.731995 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727512    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727516    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727520    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727528    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727533    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727537    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727542    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727546    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727551    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727555    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727559    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727563    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727568    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727572    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727576    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727580    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727584    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727589    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727593    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.732615 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727597    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727601    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727605    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727609    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727615    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727622    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727627    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727632    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727636    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727640    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727647    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.727651    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728562    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728572    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728577    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728583    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728590    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728595    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728599    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.733260 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728603    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728607    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728612    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728616    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728620    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728624    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728629    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728633    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728638    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728644    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728651    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728656    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728661    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728665    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728669    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728674    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728679    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728684    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728688    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.733808 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728692    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728699    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728705    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728710    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728715    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728719    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728723    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728728    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728732    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728736    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728742    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728746    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728751    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728755    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728759    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728763    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728767    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728771    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728776    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728780    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.734336 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728784    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728788    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728792    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728796    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728800    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728805    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728809    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728813    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728817    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728821    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728843    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728849    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728853    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728857    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728862    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728867    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728872    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728876    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728881    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728885    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.734894 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728889    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728894    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728900    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728904    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728908    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728912    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728916    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728920    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728925    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728929    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728933    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728937    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728941    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728945    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728948    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728953    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728957    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728961    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728965    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728969    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.735377 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728974    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728978    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728982    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728987    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728992    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.728997    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729001    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729005    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729009    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729013    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729018    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729022    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729026    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729032    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729038    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729043    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729047    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729051    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729055    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.729062    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.735873 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730041    2567 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730068    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730086    2567 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730095    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730102    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730107    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730115    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730128    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730133    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730138    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730144    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730149    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730154    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730158    2567 flags.go:64] FLAG: --cgroup-root=""
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730163    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730169    2567 flags.go:64] FLAG: --client-ca-file=""
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730173    2567 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730180    2567 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730192    2567 flags.go:64] FLAG: --cluster-domain=""
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730197    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730203    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730209    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730215    2567 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730221    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 20:59:37.736371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730226    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730232    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730237    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730242    2567 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730247    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730252    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730257    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730262    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730269    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730274    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730279    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730284    2567 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730289    2567 flags.go:64] FLAG: --enable-server="true"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730293    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730301    2567 flags.go:64] FLAG: --event-burst="100"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730306    2567 flags.go:64] FLAG: --event-qps="50"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730311    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730316    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730321    2567 flags.go:64] FLAG: --eviction-hard=""
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730328    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730333    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730338    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730342    2567 flags.go:64] FLAG: --eviction-soft=""
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730348    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730353    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 20:59:37.736938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730360    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730366    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730370    2567 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730375    2567 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730380    2567 flags.go:64] FLAG: --feature-gates=""
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730387    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730392    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730397    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730403    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730408    2567 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730413    2567 flags.go:64] FLAG: --help="false"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730417    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-140-179.ec2.internal"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730422    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730427    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730432    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730438    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730444    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730448    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730453    2567 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730458    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730462    2567 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730467    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730472    2567 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730477    2567 flags.go:64] FLAG: --kube-reserved=""
Oct 30 20:59:37.737621 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730482    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730486    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730491    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730496    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730501    2567 flags.go:64] FLAG: --lock-file=""
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730505    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730510    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730515    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730527    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730531    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730536    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730541    2567 flags.go:64] FLAG: --logging-format="text"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730546    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730551    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730556    2567 flags.go:64] FLAG: --manifest-url=""
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730561    2567 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730568    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730573    2567 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730580    2567 flags.go:64] FLAG: --max-pods="110"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730585    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730590    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730595    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730600    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730605    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730610    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 20:59:37.738206 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730616    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730627    2567 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730632    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730637    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730643    2567 flags.go:64] FLAG: --pod-cidr=""
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730647    2567 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730652    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730656    2567 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730661    2567 flags.go:64] FLAG: --port="10250"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730666    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730670    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b30b1ea0f871ada"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730676    2567 flags.go:64] FLAG: --qos-reserved=""
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730681    2567 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730685    2567 flags.go:64] FLAG: --register-node="true"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730690    2567 flags.go:64] FLAG: --register-with-taints=""
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730698    2567 flags.go:64] FLAG: --registry-burst="10"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730703    2567 flags.go:64] FLAG: --registry-qps="5"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730707    2567 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730713    2567 flags.go:64] FLAG: --reserved-memory=""
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730719    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730724    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730729    2567 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730734    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730739    2567 flags.go:64] FLAG: --runonce="false"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730744    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730749    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730753    2567 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 20:59:37.738806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730758    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730763    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730768    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730773    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730778    2567 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730783    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730787    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730791    2567 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730796    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730801    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730806    2567 flags.go:64] FLAG: --system-cgroups=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730810    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730819    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730823    2567 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730848    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730855    2567 flags.go:64] FLAG: --tls-min-version=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730859    2567 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730863    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730868    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730873    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730879    2567 flags.go:64] FLAG: --v="2"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730887    2567 flags.go:64] FLAG: --version="false"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730893    2567 flags.go:64] FLAG: --vmodule=""
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730900    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.730906    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 20:59:37.739458 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731133    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731141    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731146    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731151    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731156    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731160    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731164    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731168    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731173    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731177    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731181    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731187    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731192    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731199    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731203    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731208    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731214    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731221    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731227    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.740156 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731231    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731236    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731244    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731248    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731254    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731258    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731262    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731266    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731272    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731277    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731281    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731285    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731289    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731295    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731299    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731303    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731307    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731311    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731316    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731320    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.740608 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731324    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731328    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731332    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731336    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731340    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731344    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731348    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731352    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731356    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731361    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731365    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731369    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731373    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731377    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731383    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731386    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731390    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731395    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731399    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731404    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.741094 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731410    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731414    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731418    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731423    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731427    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731431    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731435    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731440    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731444    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731449    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731453    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731457    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731461    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731465    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731469    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731473    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731477    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731481    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731485    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731489    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.741562 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731493    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731498    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731502    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731506    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731510    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731514    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731520    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731524    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731529    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731533    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731537    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731541    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731547    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731551    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731555    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731559    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731563    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731568    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731571    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.742179 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731576    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731580    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731584    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731587    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731591    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731594    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731599    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731603    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.731647    2567 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731700    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731707    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731712    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731716    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731720    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731724    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731728    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.742807 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731733    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731737    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731741    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731747    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731752    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731756    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731760    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731764    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731768    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731772    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731778    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731782    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731787    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731791    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731796    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731800    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731804    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731810    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731815    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731819    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.743397 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731823    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731846    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731850    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731855    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731859    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731863    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731868    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731872    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731876    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731881    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731884    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731889    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731893    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731898    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731902    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731909    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731913    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731917    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731922    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.743891 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731927    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731931    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731935    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731941    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731945    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731949    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731954    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731958    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731962    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731967    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731971    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731976    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731981    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731985    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731990    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731994    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.731998    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732002    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732005    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.744349 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732010    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732014    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732018    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732022    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732027    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732031    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732036    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732040    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732044    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732050    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732054    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732058    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732062    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732066    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732071    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732075    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732081    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732088    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732092    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.744846 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732099    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732104    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732110    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732117    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732122    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732128    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732133    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732138    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732142    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732146    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732153    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732157    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732161    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732166    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732170    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732174    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732178    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732182    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732187    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.745337 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732191    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732195    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.732199    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.739183    2567 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.739199    2567 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.739216    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.739221    2567 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739299    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739304    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739307    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739310    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739313    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739316    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739318    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739321    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739323    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739326    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739329    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739332    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739340    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.745873 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739343    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739347    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739349    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739352    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739355    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739358    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739361    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739365    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739368    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739371    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739374    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739377    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739380    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739382    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739385    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739387    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739390    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739393    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739395    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739398    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.746332 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739400    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739403    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739406    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739408    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739411    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739414    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739416    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739418    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739421    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739423    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739426    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739429    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739437    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739440    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739443    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739447    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739451    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739454    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739457    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.746800 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739460    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739463    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739465    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739468    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739471    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739474    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739476    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739479    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739482    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739484    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739487    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739489    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739492    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739494    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739496    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739499    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739501    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739504    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739506    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.747311 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739509    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739511    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739513    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739516    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739518    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739521    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739523    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739532    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739535    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739537    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739539    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739542    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739545    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739547    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739550    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739552    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739555    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739557    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739560    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739562    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.747797 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739564    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739567    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739569    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739572    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739574    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739577    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739579    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739581    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739584    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739586    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739588    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739591    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739593    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739596    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739598    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.739626    2567 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:59:37.748279 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739842    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739848    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739850    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739854    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739863    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739866    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739869    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739872    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739874    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739877    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739880    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739882    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739885    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739887    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739891    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739895    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739900    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739903    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739906    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.748638 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739909    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739911    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739914    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739916    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739918    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739921    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739923    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739926    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739928    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739931    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739933    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739936    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739938    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739940    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739943    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739945    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739948    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739950    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739959    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739961    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.749145 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739964    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739967    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739969    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739971    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739974    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739976    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739979    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739981    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739984    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739986    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739989    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739991    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739993    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739996    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.739999    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740001    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740003    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740006    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740008    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740011    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.749617 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740013    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740015    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740018    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740020    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740022    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740025    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740028    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740030    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740033    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740035    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740038    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740047    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740050    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740053    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740055    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740057    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740060    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740062    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740064    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.750100 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740067    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740069    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740072    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740074    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740077    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740079    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740082    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740084    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740086    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740088    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740091    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740093    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740096    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740098    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740100    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740103    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740105    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740108    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740110    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.750545 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740112    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740115    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740117    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740120    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740122    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740125    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740133    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740135    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740138    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.740161    2567 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740215    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740219    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740222    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740225    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740228    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740230    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:59:37.751027 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740233    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740235    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740237    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740240    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740242    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740245    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740247    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740250    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740252    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740254    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740257    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740259    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740261    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740264    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740266    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740269    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740271    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740273    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740276    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:59:37.751396 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740279    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740281    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740284    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740286    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740295    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740298    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740300    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740303    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740305    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740307    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740310    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740312    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740315    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740317    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740319    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740322    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740324    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740328    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740332    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:59:37.751953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740336    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740339    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740347    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740350    2567 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740352    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740355    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740357    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740360    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740363    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740365    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740368    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740370    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740374    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740376    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740379    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740381    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740383    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740386    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740394    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740397    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:59:37.752484 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740400    2567 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740402    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740404    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740407    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740409    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740412    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740414    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740417    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740419    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740421    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740424    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740426    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740428    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740431    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740435    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740437    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740440    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740443    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740445    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740448    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:59:37.753016 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740450    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740453    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740455    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740457    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740461    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740464    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740466    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740469    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740471    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740474    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740476    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740486    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740489    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740491    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740494    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740496    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740500    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740502    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740505    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:59:37.753499 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740507    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740509    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:37.740512    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.740566    2567 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.744484    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.745595    2567 server.go:1014] "Starting client certificate rotation"
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.745684    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.746254    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.747703    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:59:37.753953 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.749059    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:59:37.759613 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.759598    2567 log.go:25] "Validated CRI v1 runtime API"
Oct 30 20:59:37.765048 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.765028    2567 log.go:25] "Validated CRI v1 image API"
Oct 30 20:59:37.765976 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.765945    2567 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 20:59:37.771159 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.771133    2567 fs.go:135] Filesystem UUIDs: map[3a30494f-d12a-4d42-b5b1-b56af3b106e8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9650cb35-4222-49ff-b6d6-8a12efb2e6de:/dev/nvme0n1p3]
Oct 30 20:59:37.771223 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.771159    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 20:59:37.776967 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.776823    2567 manager.go:217] Machine: {Timestamp:2026-10-30 20:59:37.775501394 +0000 UTC m=+0.340390283 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104180 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bfebc498d6147af716fec012ede1f SystemUUID:ec2bfebc-498d-6147-af71-6fec012ede1f BootID:02fa2206-b817-4f89-81aa-63ed791d13a3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fb:3e:59:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fb:3e:59:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:b3:8c:4b:3c:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 20:59:37.776967 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.776964    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 20:59:37.777077 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.777056    2567 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 20:59:37.779932 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.779906    2567 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 20:59:37.780076 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.779933    2567 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-179.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 20:59:37.780124 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.780079    2567 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 20:59:37.780124 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.780087    2567 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 20:59:37.780176 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.780126    2567 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:59:37.780176 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.780145    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:59:37.780998 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.780987    2567 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 20:59:37.784542 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.784529    2567 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 20:59:37.784658 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.784648    2567 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 20:59:37.785380 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.785369    2567 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 20:59:37.785411 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.785387    2567 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 20:59:37.785411 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.785405    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 20:59:37.785463 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.785414    2567 kubelet.go:405] "Adding apiserver pod source"
Oct 30 20:59:37.785463 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.785422    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 20:59:37.786037 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.786016    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:59:37.788876 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.788852    2567 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 20:59:37.794536 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.794518    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:59:37.795256 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795242    2567 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 20:59:37.795311 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795273    2567 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 20:59:37.795436 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795424    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 20:59:37.795500 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795442    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 20:59:37.795500 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795448    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 20:59:37.795500 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795453    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 20:59:37.795500 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795459    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 20:59:37.795500 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.795465    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 20:59:37.796049 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796035    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 20:59:37.796049 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796045    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 20:59:37.796049 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796053    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 20:59:37.796171 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796059    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 20:59:37.796171 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796067    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 20:59:37.796171 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796076    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 20:59:37.796657 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796647    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 20:59:37.796657 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.796657    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 20:59:37.800616 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.800599    2567 server.go:1263] "Started kubelet"
Oct 30 20:59:37.800723 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.800641    2567 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 20:59:37.800870 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.800813    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 20:59:37.800938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.800896    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 20:59:37.801489 ip-10-0-140-179 systemd[1]: Started Kubernetes Kubelet.
Oct 30 20:59:37.802014 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.801992    2567 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 20:59:37.802817 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.802796    2567 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 20:59:37.806409 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.806384    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 20:59:37.806885 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.806865    2567 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 20:59:37.807691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.807674    2567 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 20:59:37.807915 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.807895    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:37.808283 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808262    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 20:59:37.808283 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808277    2567 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 20:59:37.808458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808291    2567 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 20:59:37.808458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808379    2567 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 20:59:37.808458 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808388    2567 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 20:59:37.808682 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808667    2567 factory.go:55] Registering systemd factory
Oct 30 20:59:37.808682 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808683    2567 factory.go:223] Registration of the systemd container factory successfully
Oct 30 20:59:37.808925 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808914    2567 factory.go:153] Registering CRI-O factory
Oct 30 20:59:37.808966 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808926    2567 factory.go:223] Registration of the crio container factory successfully
Oct 30 20:59:37.809005 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808982    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 20:59:37.809005 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.808997    2567 factory.go:103] Registering Raw factory
Oct 30 20:59:37.809076 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.809011    2567 manager.go:1199] Started watching for new ooms in manager
Oct 30 20:59:37.809946 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.809931    2567 manager.go:319] Starting recovery of all containers
Oct 30 20:59:37.821854 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.821817    2567 manager.go:324] Recovery completed
Oct 30 20:59:37.826086 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.826070    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:37.827969 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.827934    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:37.828057 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.827987    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:37.828057 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.828003    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:37.828572 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.828557    2567 cpu_manager.go:226] "Starting" policy="none"
Oct 30 20:59:37.828636 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.828572    2567 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 20:59:37.828636 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.828593    2567 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 20:59:37.828636 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.828564    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:37.829254 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.829239    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:37.829319 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.829270    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:37.829319 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.829285    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:37.830734 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.830721    2567 policy_none.go:50] "Start"
Oct 30 20:59:37.830778 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.830738    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 20:59:37.830778 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.830748    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 20:59:37.832688 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.832674    2567 policy_none.go:44] "Start"
Oct 30 20:59:37.833118 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.833081    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-179.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:59:37.833200 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.833182    2567 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-179.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 20:59:37.834040 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.833081    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-179.ec2.internal.18e36a522d1e16ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-179.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-179.ec2.internal,},FirstTimestamp:2026-10-30 20:59:37.800398591 +0000 UTC m=+0.365287481,LastTimestamp:2026-10-30 20:59:37.800398591 +0000 UTC m=+0.365287481,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-179.ec2.internal,}"
Oct 30 20:59:37.842201 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.842180    2567 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 20:59:37.842294 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.842274    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-svhsk"
Oct 30 20:59:37.842347 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.842299    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-svhsk"
Oct 30 20:59:37.849065 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.848993    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-179.ec2.internal.18e36a522ec2c46a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-179.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-179.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-179.ec2.internal,},FirstTimestamp:2026-10-30 20:59:37.827968106 +0000 UTC m=+0.392856995,LastTimestamp:2026-10-30 20:59:37.827968106 +0000 UTC m=+0.392856995,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-179.ec2.internal,}"
Oct 30 20:59:37.849236 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.849172    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 20:59:37.849236 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.849174    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 20:59:37.849324 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.849257    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-179.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 20:59:37.873568 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.873527    2567 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 20:59:37.876239 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876219    2567 manager.go:341] "Starting Device Plugin manager"
Oct 30 20:59:37.876344 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.876252    2567 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 20:59:37.876344 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876265    2567 server.go:87] "Starting device plugin registration server"
Oct 30 20:59:37.876546 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876534    2567 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 20:59:37.876577 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876550    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 20:59:37.876669 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876646    2567 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 20:59:37.876775 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876743    2567 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 20:59:37.876775 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.876751    2567 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 20:59:37.877305 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.877285    2567 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 20:59:37.877402 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.877339    2567 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:37.960150 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.960110    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 20:59:37.962048 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.961515    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 20:59:37.962048 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.961540    2567 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 20:59:37.962048 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.961572    2567 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 20:59:37.962048 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.961614    2567 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 20:59:37.974156 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.974085    2567 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:59:37.977152 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.977137    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:37.978124 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.978106    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:37.978232 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.978139    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:37.978232 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.978154    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:37.978232 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.978184    2567 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-179.ec2.internal"
Oct 30 20:59:37.994043 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:37.994016    2567 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-179.ec2.internal"
Oct 30 20:59:37.994043 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:37.994045    2567 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-179.ec2.internal\": node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.043726 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.043699    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.062413 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.062369    2567 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"]
Oct 30 20:59:38.062565 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.062462    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:38.063515 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.063497    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:38.063589 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.063529    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:38.063589 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.063538    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:38.065624 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.065612    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:38.066179 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.066236 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066197    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:38.066385 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066365    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:38.066461 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066404    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:38.066461 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066420    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:38.066895 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066884    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:38.066957 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066908    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:38.066957 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.066917    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:38.068555 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.068539    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.068631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.068569    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:38.069309 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.069294    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:38.069386 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.069334    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:38.069386 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.069349    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:38.105525 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.105504    2567 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-179.ec2.internal\" not found" node="ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.108123 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.108106    2567 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-179.ec2.internal\" not found" node="ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.109555 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.109537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.109599 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.109563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.109599 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.109581    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d94f708ebb6ce685e97c708d7e6cc9d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" (UID: \"0d94f708ebb6ce685e97c708d7e6cc9d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.144256 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.144219    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.210141 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.210104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.210141 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.210143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.210346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.210162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d94f708ebb6ce685e97c708d7e6cc9d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" (UID: \"0d94f708ebb6ce685e97c708d7e6cc9d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.210346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.210206    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.210346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.210207    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d94f708ebb6ce685e97c708d7e6cc9d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" (UID: \"0d94f708ebb6ce685e97c708d7e6cc9d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.210346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.210213    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.245390 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.245330    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.345493 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.345462    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.408723 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.408703    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.411857 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.411813    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.446710 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.446508    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.547143 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.547048    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.647683 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.647641    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.746294 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.746247    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 20:59:38.746784 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.746414    2567 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 20:59:38.748426 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.748405    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.846980 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.846937    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 20:54:38 +0000 UTC" deadline="2028-05-07 22:31:55.973792933 +0000 UTC"
Oct 30 20:59:38.846980 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.846972    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13321h32m17.126825696s"
Oct 30 20:59:38.849096 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:38.849075    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 20:59:38.853243 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.853221    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:59:38.854239 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.854221    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 20:59:38.854346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.854257    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 20:59:38.854346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.854272    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 20:59:38.872052 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.872023    2567 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:59:38.905228 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.905198    2567 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:59:38.908241 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.908223    2567 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.936813 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.936785    2567 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:59:38.937800 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.937782    2567 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 20:59:38.946709 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.946675    2567 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:59:38.960431 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:38.960405    2567 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:59:39.068882 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:39.068839    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71628ad4075ef9e5c4cd365578233f56.slice/crio-1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef WatchSource:0}: Error finding container 1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef: Status 404 returned error can't find the container with id 1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef
Oct 30 20:59:39.069119 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:39.069094    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d94f708ebb6ce685e97c708d7e6cc9d.slice/crio-4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193 WatchSource:0}: Error finding container 4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193: Status 404 returned error can't find the container with id 4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193
Oct 30 20:59:39.075472 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.075455    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:59:39.667154 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.667126    2567 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:59:39.786849 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.786799    2567 apiserver.go:52] "Watching apiserver"
Oct 30 20:59:39.808001 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.807967    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 20:59:39.836295 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.836247    2567 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 20:59:39.836706 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.836658    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal","openshift-dns/node-resolver-5wsmb","openshift-image-registry/node-ca-lb6cv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qlqw5","openshift-multus/multus-additional-cni-plugins-mmfcw","openshift-network-diagnostics/network-check-target-hkm5p","openshift-network-operator/iptables-alerter-rcqw6","kube-system/global-pull-secret-syncer-cpdvw","kube-system/konnectivity-agent-8t9kn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q","openshift-multus/multus-22z8l","openshift-multus/network-metrics-daemon-8pcqj","openshift-ovn-kubernetes/ovnkube-node-d44wt"]
Oct 30 20:59:39.839749 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.839721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:39.842387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.842353    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:39.842513 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:39.842443    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:39.844816 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.844793    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:39.848302 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.848248    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.851713 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.851687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.854732 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.854712    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.860622 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.860587    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tzc2p\""
Oct 30 20:59:39.860760 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.860731    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:39.860842 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.860788    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 20:59:39.860842 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:39.860821    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:39.860977 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.860838    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 20:59:39.860977 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.860946    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:39.861071 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:39.861003    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:39.864300 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.863465    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:39.866244 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.866226    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:39.869596 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.869572    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.870861 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.870755    2567 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 20:59:39.870861 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.870826    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rdd9l"
Oct 30 20:59:39.871760 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.871743    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.910532 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.910504    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 20:59:39.919478 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919408    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-multus\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.919478 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-cnibin\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.919478 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4z66\" (UniqueName: \"kubernetes.io/projected/3b759b7b-3f86-427d-a1f7-6382fbe96464-kube-api-access-w4z66\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919487    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-device-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-lib-modules\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-netns\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-conf-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81eb3986-da71-4e0e-862b-c84d3f33815c-konnectivity-ca\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919581    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-systemd-units\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919597    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-log-socket\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919612    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-netd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-cni-binary-copy\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-bin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81eb3986-da71-4e0e-862b-c84d3f33815c-agent-certs\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:39.919691 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc96670-a561-4f89-896a-a1b8369560a8-iptables-alerter-script\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-var-lib-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-etc-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-script-lib\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919757    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-kubernetes\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919787    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-registration-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-slash\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-ovn\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56x7c\" (UniqueName: \"kubernetes.io/projected/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-kube-api-access-56x7c\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysconfig\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919958    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-tmp\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.919989    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-kubelet-config\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-system-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-multus-certs\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.920291 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920040    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-kubelet\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-etc-kubernetes\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqlkc\" (UniqueName: \"kubernetes.io/projected/a7034506-9c4f-4252-8564-31d0bbe36a16-kube-api-access-mqlkc\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-system-cni-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920177    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b759b7b-3f86-427d-a1f7-6382fbe96464-tmp-dir\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920219    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-socket-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-etc-selinux\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920276    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovn-node-metrics-cert\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-dbus\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920348    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-cnibin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdkm\" (UniqueName: \"kubernetes.io/projected/458e6a65-37ba-42b3-a21a-a411d302093c-kube-api-access-ftdkm\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-systemd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-conf\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.921192 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920597    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxj6c\" (UniqueName: \"kubernetes.io/projected/b0cf3a14-78d5-4493-a209-25a4f0233d59-kube-api-access-pxj6c\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b346fb18-266d-44b4-998e-afdaf5ae0085-host\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-sys\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920706    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-kubelet\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc96670-a561-4f89-896a-a1b8369560a8-host-slash\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-799cl\" (UniqueName: \"kubernetes.io/projected/4fc96670-a561-4f89-896a-a1b8369560a8-kube-api-access-799cl\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-env-overrides\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-run\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnvzb\" (UniqueName: \"kubernetes.io/projected/6ef41467-164c-4bc7-8870-d44ec9c655a9-kube-api-access-hnvzb\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.920983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5n8v\" (UniqueName: \"kubernetes.io/projected/b346fb18-266d-44b4-998e-afdaf5ae0085-kube-api-access-w5n8v\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-config\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-systemd\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921147    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-var-lib-kubelet\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922122 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-k8s-cni-cncf-io\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-hostroot\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-daemon-config\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921435    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-netns\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921592    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-tuned\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvdsw\" (UniqueName: \"kubernetes.io/projected/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kube-api-access-bvdsw\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b759b7b-3f86-427d-a1f7-6382fbe96464-hosts-file\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-sys-fs\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921902    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-host\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921930    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b346fb18-266d-44b4-998e-afdaf5ae0085-serviceca\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-bin\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.922820 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.921982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-modprobe-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:39.923356 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.922039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-node-log\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:39.923356 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.922069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-socket-dir-parent\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:39.967201 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967138    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal" event={"ID":"0d94f708ebb6ce685e97c708d7e6cc9d","Type":"ContainerStarted","Data":"4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193"}
Oct 30 20:59:39.967426 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967325    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 20:59:39.967631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967590    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 20:59:39.967631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967633    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:59:39.967631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967658    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 20:59:39.967938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967707    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.967938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967739    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 20:59:39.967938 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967770    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:59:39.968107 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.967937    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tnqjk\""
Oct 30 20:59:39.968107 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968036    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.968107 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968052    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.968337 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968109    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 20:59:39.968337 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968119    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 20:59:39.968337 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968190    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.968337 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968324    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 20:59:39.968540 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968458    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 20:59:39.968540 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968473    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j8v8x\""
Oct 30 20:59:39.968540 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968527    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968541    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-79n9v\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968592    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968593    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-hvtl2\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968603    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968628    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968646    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968677    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968689    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 20:59:39.968689 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968694    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.969155 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968712    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 20:59:39.969155 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.968790    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gszwc\""
Oct 30 20:59:39.969260 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.969207    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" event={"ID":"71628ad4075ef9e5c4cd365578233f56","Type":"ContainerStarted","Data":"1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef"}
Oct 30 20:59:39.983766 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.983718    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f7r6m\""
Oct 30 20:59:39.983939 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.983801    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4x2zw\""
Oct 30 20:59:39.983939 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.983865    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b8vzq\""
Oct 30 20:59:39.983939 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.983891    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 20:59:39.983939 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:39.983730    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 20:59:40.023141 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-system-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023141 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-multus-certs\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023171    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-kubelet\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023195    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023218    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-etc-kubernetes\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqlkc\" (UniqueName: \"kubernetes.io/projected/a7034506-9c4f-4252-8564-31d0bbe36a16-kube-api-access-mqlkc\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023262    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-system-cni-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023257    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-multus-certs\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023279    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-system-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b759b7b-3f86-427d-a1f7-6382fbe96464-tmp-dir\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023322    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-kubelet\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.023387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023381    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-etc-kubernetes\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023382    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-system-cni-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-socket-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023421    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-etc-selinux\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovn-node-metrics-cert\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023477    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-socket-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-dbus\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-cnibin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023531    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-etc-selinux\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdkm\" (UniqueName: \"kubernetes.io/projected/458e6a65-37ba-42b3-a21a-a411d302093c-kube-api-access-ftdkm\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023619    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-cnibin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023644    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b759b7b-3f86-427d-a1f7-6382fbe96464-tmp-dir\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-systemd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023725    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-dbus\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.023824 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-conf\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023777    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-systemd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023825    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023886    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxj6c\" (UniqueName: \"kubernetes.io/projected/b0cf3a14-78d5-4493-a209-25a4f0233d59-kube-api-access-pxj6c\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b346fb18-266d-44b4-998e-afdaf5ae0085-host\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023960    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-conf\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023971    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-sys\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-kubelet\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024021    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b346fb18-266d-44b4-998e-afdaf5ae0085-host\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.023998    2567 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024028    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc96670-a561-4f89-896a-a1b8369560a8-host-slash\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024076    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-sys\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024076    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-kubelet\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-799cl\" (UniqueName: \"kubernetes.io/projected/4fc96670-a561-4f89-896a-a1b8369560a8-kube-api-access-799cl\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-env-overrides\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024133    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-run\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024111    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc96670-a561-4f89-896a-a1b8369560a8-host-slash\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.024722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024153    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024187    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-run\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnvzb\" (UniqueName: \"kubernetes.io/projected/6ef41467-164c-4bc7-8870-d44ec9c655a9-kube-api-access-hnvzb\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.024267    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.024357    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret podName:4c1d2b37-38ac-4aca-90d3-144d6537006f nodeName:}" failed. No retries permitted until 2026-10-30 20:59:40.524313865 +0000 UTC m=+3.089202767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret") pod "global-pull-secret-syncer-cpdvw" (UID: "4c1d2b37-38ac-4aca-90d3-144d6537006f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5n8v\" (UniqueName: \"kubernetes.io/projected/b346fb18-266d-44b4-998e-afdaf5ae0085-kube-api-access-w5n8v\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-config\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024433    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-systemd\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024458    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-var-lib-kubelet\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-k8s-cni-cncf-io\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024500    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-systemd\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-hostroot\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-daemon-config\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024543    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-env-overrides\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024555    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-var-lib-kubelet\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.025387 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024563    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024586    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-k8s-cni-cncf-io\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024593    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024596    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-hostroot\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.024647    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.024682    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 20:59:40.52467035 +0000 UTC m=+3.089559227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-netns\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-tuned\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvdsw\" (UniqueName: \"kubernetes.io/projected/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kube-api-access-bvdsw\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b759b7b-3f86-427d-a1f7-6382fbe96464-hosts-file\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024793    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-sys-fs\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024807    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-netns\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-host\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.026253 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024875    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024892    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b346fb18-266d-44b4-998e-afdaf5ae0085-serviceca\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024927    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-sys-fs\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-bin\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024966    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-modprobe-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024977    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024936    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.024991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-node-log\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-socket-dir-parent\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025041    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-multus\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025052    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b759b7b-3f86-427d-a1f7-6382fbe96464-hosts-file\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-cnibin\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025094    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-cnibin\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4z66\" (UniqueName: \"kubernetes.io/projected/3b759b7b-3f86-427d-a1f7-6382fbe96464-kube-api-access-w4z66\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025154    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-socket-dir-parent\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-device-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025162    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-node-log\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-lib-modules\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.026877 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025195    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-netns\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025198    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-device-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-conf-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025239    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-conf-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025261    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-modprobe-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025125    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-host\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025265    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-netns\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025309    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-lib-modules\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025315    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-bin\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81eb3986-da71-4e0e-862b-c84d3f33815c-konnectivity-ca\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-systemd-units\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025333    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-multus\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-log-socket\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-netd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025447    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-systemd-units\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025456    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-cni-binary-copy\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025467    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-log-socket\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025482    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-bin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.027631 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81eb3986-da71-4e0e-862b-c84d3f33815c-agent-certs\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025525    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-netd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc96670-a561-4f89-896a-a1b8369560a8-iptables-alerter-script\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-var-lib-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-etc-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-script-lib\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-kubernetes\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-registration-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-slash\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-ovn\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56x7c\" (UniqueName: \"kubernetes.io/projected/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-kube-api-access-56x7c\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025795    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025850    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysconfig\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025848    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-var-lib-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysconfig\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.028345 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025872    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81eb3986-da71-4e0e-862b-c84d3f33815c-konnectivity-ca\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025886    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-etc-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025910    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-daemon-config\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025958    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025581    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-config\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026012    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-slash\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026024    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-bin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026070    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-ovn\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026077    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-registration-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026141    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-cni-binary-copy\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026230    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-kubernetes\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.025929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-tmp\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-kubelet-config\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026359    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-kubelet-config\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026449    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b346fb18-266d-44b4-998e-afdaf5ae0085-serviceca\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026551    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-script-lib\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026555    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc96670-a561-4f89-896a-a1b8369560a8-iptables-alerter-script\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.029047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.026588    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.030266 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.030237    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-tuned\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.033488 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.033464    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-tmp\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.034583 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.034558    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovn-node-metrics-cert\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.034725 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.034706    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81eb3986-da71-4e0e-862b-c84d3f33815c-agent-certs\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:40.045705 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.045680    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:59:40.045705 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.045705    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:59:40.045895 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.045719    2567 projected.go:196] Error preparing data for projected volume kube-api-access-pq4x6 for pod openshift-network-diagnostics/network-check-target-hkm5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:40.045895 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.045804    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6 podName:09b454b3-3ffa-4639-9fb9-5fbc06ebd0da nodeName:}" failed. No retries permitted until 2026-10-30 20:59:40.545783961 +0000 UTC m=+3.110672854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pq4x6" (UniqueName: "kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6") pod "network-check-target-hkm5p" (UID: "09b454b3-3ffa-4639-9fb9-5fbc06ebd0da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:40.053584 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.053550    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4z66\" (UniqueName: \"kubernetes.io/projected/3b759b7b-3f86-427d-a1f7-6382fbe96464-kube-api-access-w4z66\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.060215 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060175    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqlkc\" (UniqueName: \"kubernetes.io/projected/a7034506-9c4f-4252-8564-31d0bbe36a16-kube-api-access-mqlkc\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.060346 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060317    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxj6c\" (UniqueName: \"kubernetes.io/projected/b0cf3a14-78d5-4493-a209-25a4f0233d59-kube-api-access-pxj6c\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.060690 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060665    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-799cl\" (UniqueName: \"kubernetes.io/projected/4fc96670-a561-4f89-896a-a1b8369560a8-kube-api-access-799cl\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.060791 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060734    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5n8v\" (UniqueName: \"kubernetes.io/projected/b346fb18-266d-44b4-998e-afdaf5ae0085-kube-api-access-w5n8v\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.060849 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060790    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnvzb\" (UniqueName: \"kubernetes.io/projected/6ef41467-164c-4bc7-8870-d44ec9c655a9-kube-api-access-hnvzb\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:40.060900 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060862    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56x7c\" (UniqueName: \"kubernetes.io/projected/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-kube-api-access-56x7c\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.060966 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.060937    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvdsw\" (UniqueName: \"kubernetes.io/projected/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kube-api-access-bvdsw\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.062708 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.062686    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdkm\" (UniqueName: \"kubernetes.io/projected/458e6a65-37ba-42b3-a21a-a411d302093c-kube-api-access-ftdkm\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.157761 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.157722    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:40.169099 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.168879    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 20:59:40.170881 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.170803    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81eb3986_da71_4e0e_862b_c84d3f33815c.slice/crio-5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023 WatchSource:0}: Error finding container 5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023: Status 404 returned error can't find the container with id 5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023
Oct 30 20:59:40.179942 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.179907    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb346fb18_266d_44b4_998e_afdaf5ae0085.slice/crio-6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d WatchSource:0}: Error finding container 6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d: Status 404 returned error can't find the container with id 6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d
Oct 30 20:59:40.193384 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.193352    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 20:59:40.199255 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.199055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 20:59:40.205308 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.205264    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cf3a14_78d5_4493_a209_25a4f0233d59.slice/crio-1705d4ccad5ac18b6bf1add2c0275b8c515fc8ea3f4612a4638b13d352c317f0 WatchSource:0}: Error finding container 1705d4ccad5ac18b6bf1add2c0275b8c515fc8ea3f4612a4638b13d352c317f0: Status 404 returned error can't find the container with id 1705d4ccad5ac18b6bf1add2c0275b8c515fc8ea3f4612a4638b13d352c317f0
Oct 30 20:59:40.207964 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.207943    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5wsmb"
Oct 30 20:59:40.210877 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.210841    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddabeb57c_4fde_4a6d_a4c8_e1943641d0aa.slice/crio-2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6 WatchSource:0}: Error finding container 2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6: Status 404 returned error can't find the container with id 2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6
Oct 30 20:59:40.215289 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.215079    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 20:59:40.217780 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.217757    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 20:59:40.222017 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.221986    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b759b7b_3f86_427d_a1f7_6382fbe96464.slice/crio-0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a WatchSource:0}: Error finding container 0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a: Status 404 returned error can't find the container with id 0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a
Oct 30 20:59:40.229437 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.229406    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458e6a65_37ba_42b3_a21a_a411d302093c.slice/crio-82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7 WatchSource:0}: Error finding container 82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7: Status 404 returned error can't find the container with id 82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7
Oct 30 20:59:40.230414 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.230387    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc96670_a561_4f89_896a_a1b8369560a8.slice/crio-8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68 WatchSource:0}: Error finding container 8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68: Status 404 returned error can't find the container with id 8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68
Oct 30 20:59:40.232229 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.232217    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 20:59:40.236654 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.236585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-22z8l"
Oct 30 20:59:40.247706 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.247658    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7fa2f1_b578_4a7c_87d6_3b2d018e6b22.slice/crio-0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8 WatchSource:0}: Error finding container 0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8: Status 404 returned error can't find the container with id 0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8
Oct 30 20:59:40.248879 ip-10-0-140-179 kubenswrapper[2567]: W1030 20:59:40.248822    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7034506_9c4f_4252_8564_31d0bbe36a16.slice/crio-58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b WatchSource:0}: Error finding container 58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b: Status 404 returned error can't find the container with id 58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b
Oct 30 20:59:40.529772 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.529736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:40.529772 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.529782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:40.530043 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.529918    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:40.530043 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.529995    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret podName:4c1d2b37-38ac-4aca-90d3-144d6537006f nodeName:}" failed. No retries permitted until 2026-10-30 20:59:41.52997934 +0000 UTC m=+4.094868218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret") pod "global-pull-secret-syncer-cpdvw" (UID: "4c1d2b37-38ac-4aca-90d3-144d6537006f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:40.530043 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.529922    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:40.530184 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.530086    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 20:59:41.530066329 +0000 UTC m=+4.094955228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:40.633188 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.633149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:40.633386 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.633364    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:59:40.633431 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.633395    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:59:40.633431 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.633410    2567 projected.go:196] Error preparing data for projected volume kube-api-access-pq4x6 for pod openshift-network-diagnostics/network-check-target-hkm5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:40.633506 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:40.633488    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6 podName:09b454b3-3ffa-4639-9fb9-5fbc06ebd0da nodeName:}" failed. No retries permitted until 2026-10-30 20:59:41.633466051 +0000 UTC m=+4.198354951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pq4x6" (UniqueName: "kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6") pod "network-check-target-hkm5p" (UID: "09b454b3-3ffa-4639-9fb9-5fbc06ebd0da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:40.874840 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.874684    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:54:40 +0000 UTC" deadline="2028-05-05 05:04:52.104357086 +0000 UTC"
Oct 30 20:59:40.874840 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.874774    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13256h5m11.229587881s"
Oct 30 20:59:40.979036 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.978874    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerStarted","Data":"82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7"}
Oct 30 20:59:40.983492 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.983338    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6"}
Oct 30 20:59:40.987370 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.987335    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qlqw5" event={"ID":"b0cf3a14-78d5-4493-a209-25a4f0233d59","Type":"ContainerStarted","Data":"1705d4ccad5ac18b6bf1add2c0275b8c515fc8ea3f4612a4638b13d352c317f0"}
Oct 30 20:59:40.989757 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.989725    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lb6cv" event={"ID":"b346fb18-266d-44b4-998e-afdaf5ae0085","Type":"ContainerStarted","Data":"6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d"}
Oct 30 20:59:40.993259 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:40.993227    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal" event={"ID":"0d94f708ebb6ce685e97c708d7e6cc9d","Type":"ContainerStarted","Data":"8910d803d0952c25886cc5aa10f745e07273b99824ea1a2572bf36e88e42be64"}
Oct 30 20:59:41.001623 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.001595    2567 generic.go:356] "Generic (PLEG): container finished" podID="71628ad4075ef9e5c4cd365578233f56" containerID="7a7139692a4795fa3b6488f3d742c1d53df696ecfc1e0b30c5c2a61ed702b640" exitCode=0
Oct 30 20:59:41.001760 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.001685    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" event={"ID":"71628ad4075ef9e5c4cd365578233f56","Type":"ContainerDied","Data":"7a7139692a4795fa3b6488f3d742c1d53df696ecfc1e0b30c5c2a61ed702b640"}
Oct 30 20:59:41.007924 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.007873    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-22z8l" event={"ID":"a7034506-9c4f-4252-8564-31d0bbe36a16","Type":"ContainerStarted","Data":"58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b"}
Oct 30 20:59:41.017949 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.017912    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5wsmb" event={"ID":"3b759b7b-3f86-427d-a1f7-6382fbe96464","Type":"ContainerStarted","Data":"0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a"}
Oct 30 20:59:41.025682 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.025578    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8t9kn" event={"ID":"81eb3986-da71-4e0e-862b-c84d3f33815c","Type":"ContainerStarted","Data":"5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023"}
Oct 30 20:59:41.051029 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.050653    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q" event={"ID":"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22","Type":"ContainerStarted","Data":"0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8"}
Oct 30 20:59:41.059026 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.058979    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rcqw6" event={"ID":"4fc96670-a561-4f89-896a-a1b8369560a8","Type":"ContainerStarted","Data":"8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68"}
Oct 30 20:59:41.080696 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.079247    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal" podStartSLOduration=2.079222927 podStartE2EDuration="2.079222927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:41.037002031 +0000 UTC m=+3.601890934" watchObservedRunningTime="2026-10-30 20:59:41.079222927 +0000 UTC m=+3.644112026"
Oct 30 20:59:41.543107 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.542226    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:41.543107 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.542285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:41.543107 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.542453    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:41.543107 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.542517    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 20:59:43.54249772 +0000 UTC m=+6.107386616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:41.543107 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.542957    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:41.543107 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.543007    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret podName:4c1d2b37-38ac-4aca-90d3-144d6537006f nodeName:}" failed. No retries permitted until 2026-10-30 20:59:43.542992174 +0000 UTC m=+6.107881056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret") pod "global-pull-secret-syncer-cpdvw" (UID: "4c1d2b37-38ac-4aca-90d3-144d6537006f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:41.643213 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.643162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:41.643428 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.643348    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:59:41.643428 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.643371    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:59:41.643428 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.643385    2567 projected.go:196] Error preparing data for projected volume kube-api-access-pq4x6 for pod openshift-network-diagnostics/network-check-target-hkm5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:41.643590 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.643448    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6 podName:09b454b3-3ffa-4639-9fb9-5fbc06ebd0da nodeName:}" failed. No retries permitted until 2026-10-30 20:59:43.643429772 +0000 UTC m=+6.208318654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pq4x6" (UniqueName: "kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6") pod "network-check-target-hkm5p" (UID: "09b454b3-3ffa-4639-9fb9-5fbc06ebd0da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:41.875551 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.875500    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:54:40 +0000 UTC" deadline="2028-07-16 07:21:00.539314564 +0000 UTC"
Oct 30 20:59:41.875551 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.875550    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14986h21m18.663769838s"
Oct 30 20:59:41.962304 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.962270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:41.962507 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.962424    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:41.962913 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.962874    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:41.963012 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.962993    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:41.963184 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:41.963164    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:41.963288 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:41.963267    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:42.089077 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:42.089041    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" event={"ID":"71628ad4075ef9e5c4cd365578233f56","Type":"ContainerStarted","Data":"3998a26c86bd35061af5d9941e158ea037c43a8b63f2541a0f474e696e666626"}
Oct 30 20:59:43.101688 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.100350    2567 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="f6e03aa01360d117a80d1f8e39d6d064cac39808ec98e128e71bc9a128db0ddf" exitCode=0
Oct 30 20:59:43.101688 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.101427    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerDied","Data":"f6e03aa01360d117a80d1f8e39d6d064cac39808ec98e128e71bc9a128db0ddf"}
Oct 30 20:59:43.177539 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.177471    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" podStartSLOduration=4.1774485630000004 podStartE2EDuration="4.177448563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:42.132455713 +0000 UTC m=+4.697344617" watchObservedRunningTime="2026-10-30 20:59:43.177448563 +0000 UTC m=+5.742337466"
Oct 30 20:59:43.561279 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.561185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:43.561279 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.561246    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:43.561531 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.561397    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:43.561531 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.561479    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 20:59:47.561459618 +0000 UTC m=+10.126348512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:43.561932 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.561896    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:43.562034 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.561967    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret podName:4c1d2b37-38ac-4aca-90d3-144d6537006f nodeName:}" failed. No retries permitted until 2026-10-30 20:59:47.56194975 +0000 UTC m=+10.126838644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret") pod "global-pull-secret-syncer-cpdvw" (UID: "4c1d2b37-38ac-4aca-90d3-144d6537006f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:43.662970 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.662235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:43.662970 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.662464    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:59:43.662970 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.662489    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:59:43.662970 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.662502    2567 projected.go:196] Error preparing data for projected volume kube-api-access-pq4x6 for pod openshift-network-diagnostics/network-check-target-hkm5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:43.662970 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.662567    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6 podName:09b454b3-3ffa-4639-9fb9-5fbc06ebd0da nodeName:}" failed. No retries permitted until 2026-10-30 20:59:47.662549581 +0000 UTC m=+10.227438476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pq4x6" (UniqueName: "kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6") pod "network-check-target-hkm5p" (UID: "09b454b3-3ffa-4639-9fb9-5fbc06ebd0da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:43.965423 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.964088    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:43.965423 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.964214    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:43.965423 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:43.964253    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:43.965423 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.964376    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:43.965423 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.964516    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:43.965423 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:43.964602    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:44.116814 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:44.116718    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8t9kn" event={"ID":"81eb3986-da71-4e0e-862b-c84d3f33815c","Type":"ContainerStarted","Data":"52678a80781f9bbaeb80c74d95b4e88d207dbc9c5e1bd1017b486ac56ad1220c"}
Oct 30 20:59:45.163944 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:45.163864    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8t9kn" podStartSLOduration=3.5326364249999997 podStartE2EDuration="7.163846278s" totalImagesPullingTime="3.631209853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:45.163253984 +0000 UTC m=+7.728142889" watchObservedRunningTime="2026-10-30 20:59:45.163846278 +0000 UTC m=+7.728735179"
Oct 30 20:59:45.967508 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:45.967417    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:45.967508 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:45.967490    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:45.967757 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:45.967636    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:45.967757 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:45.967732    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:45.967900 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:45.967866    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:45.967972 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:45.967952    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:46.127542 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:46.127509    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lb6cv" event={"ID":"b346fb18-266d-44b4-998e-afdaf5ae0085","Type":"ContainerStarted","Data":"802f0dba2742c046dd085b7bae6cdbed4f63c195c7730f7773d0772f5ee6389d"}
Oct 30 20:59:46.655491 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:46.655454    2567 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:46.656371 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:46.656350    2567 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:46.695404 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:46.694144    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lb6cv" podStartSLOduration=3.234248237 podStartE2EDuration="8.694123577s" totalImagesPullingTime="5.45987534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:46.167716782 +0000 UTC m=+8.732605680" watchObservedRunningTime="2026-10-30 20:59:46.694123577 +0000 UTC m=+9.259012477"
Oct 30 20:59:47.134342 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.133861    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q" event={"ID":"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22","Type":"ContainerStarted","Data":"2e51144ffe369d56f1e17d924293e5fc4e70ab6515fb7846dbe919f7ff506a3d"}
Oct 30 20:59:47.134342 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.133926    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:47.134993 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.134776    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 20:59:47.597806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.597040    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:47.597806 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.597091    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:47.597806 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.597233    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:47.597806 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.597297    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 20:59:55.597278757 +0000 UTC m=+18.162167637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:47.597806 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.597712    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:47.597806 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.597765    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret podName:4c1d2b37-38ac-4aca-90d3-144d6537006f nodeName:}" failed. No retries permitted until 2026-10-30 20:59:55.597749764 +0000 UTC m=+18.162638644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret") pod "global-pull-secret-syncer-cpdvw" (UID: "4c1d2b37-38ac-4aca-90d3-144d6537006f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:47.698375 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.697684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:47.698375 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.697906    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:59:47.698375 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.697930    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:59:47.698375 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.697945    2567 projected.go:196] Error preparing data for projected volume kube-api-access-pq4x6 for pod openshift-network-diagnostics/network-check-target-hkm5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:47.698375 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.698008    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6 podName:09b454b3-3ffa-4639-9fb9-5fbc06ebd0da nodeName:}" failed. No retries permitted until 2026-10-30 20:59:55.697988387 +0000 UTC m=+18.262877267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pq4x6" (UniqueName: "kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6") pod "network-check-target-hkm5p" (UID: "09b454b3-3ffa-4639-9fb9-5fbc06ebd0da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:47.963492 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.963462    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:47.963620 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.963594    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:47.963894 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.963854    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:47.963992 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.963954    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:47.964057 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:47.964008    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:47.964108 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:47.964085    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:48.022220 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:48.022125    2567 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 20:59:48.156436 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:48.156281    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q" event={"ID":"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22","Type":"ContainerStarted","Data":"4940e90f66681d489dfb515918b3538a5eeadb6d86153438be86b62f50b9dcc6"}
Oct 30 20:59:48.886252 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:48.886132    2567 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T20:59:48.022145451Z","UUID":"3e628b82-65f8-46f0-9ed4-e94f2ce207b9","Handler":null,"Name":"","Endpoint":""}
Oct 30 20:59:48.887742 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:48.887713    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 20:59:48.887880 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:48.887751    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 20:59:49.160885 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.160776    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q" event={"ID":"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22","Type":"ContainerStarted","Data":"b74a2bd20f440484853b5b64a8b8d6292681723623af1175cf5f3fd156817d89"}
Oct 30 20:59:49.163663 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.162992    2567 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="d5e8d75c309b375a5724f95ddb8fcad42f012bc1d72f6c87a5d7d0dcded62f7b" exitCode=0
Oct 30 20:59:49.163663 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.163086    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerDied","Data":"d5e8d75c309b375a5724f95ddb8fcad42f012bc1d72f6c87a5d7d0dcded62f7b"}
Oct 30 20:59:49.165865 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.165816    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-22z8l" event={"ID":"a7034506-9c4f-4252-8564-31d0bbe36a16","Type":"ContainerStarted","Data":"49ad97258abb978e464a0b9016290fbc0a0847a6980e59e30f285776f67a7c3d"}
Oct 30 20:59:49.168463 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.168439    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5wsmb" event={"ID":"3b759b7b-3f86-427d-a1f7-6382fbe96464","Type":"ContainerStarted","Data":"c6188bb20025f89ca7f8548526cf8b8f03402b992796d02fa4650a2e64d8a471"}
Oct 30 20:59:49.203666 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.203608    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q" podStartSLOduration=2.78893173 podStartE2EDuration="11.203593509s" totalImagesPullingTime="8.414661779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:49.203073836 +0000 UTC m=+11.767962735" watchObservedRunningTime="2026-10-30 20:59:49.203593509 +0000 UTC m=+11.768482441"
Oct 30 20:59:49.295768 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.295532    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-22z8l" podStartSLOduration=3.137705129 podStartE2EDuration="11.295512294s" totalImagesPullingTime="8.157807165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:49.295061177 +0000 UTC m=+11.859950075" watchObservedRunningTime="2026-10-30 20:59:49.295512294 +0000 UTC m=+11.860401193"
Oct 30 20:59:49.335421 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.335366    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-5wsmb" podStartSLOduration=3.154701539 podStartE2EDuration="11.335349088s" totalImagesPullingTime="8.180647549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:49.334443577 +0000 UTC m=+11.899332490" watchObservedRunningTime="2026-10-30 20:59:49.335349088 +0000 UTC m=+11.900237986"
Oct 30 20:59:49.965887 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.965806    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:49.966362 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:49.965959    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:49.966362 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.966297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:49.966504 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:49.966400    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:49.966504 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:49.966446    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:49.966504 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:49.966485    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:50.173505 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:50.172762    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rcqw6" event={"ID":"4fc96670-a561-4f89-896a-a1b8369560a8","Type":"ContainerStarted","Data":"9cfecfd3f23b47b118174149780d8f4957fe162a228cc0278067e41e764a31b9"}
Oct 30 20:59:50.176466 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:50.176429    2567 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="e7d61e03eca0eb16b88f70a8a2dee17ab9375460c2e9b4bd11de86569158caa5" exitCode=0
Oct 30 20:59:50.177592 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:50.177565    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerDied","Data":"e7d61e03eca0eb16b88f70a8a2dee17ab9375460c2e9b4bd11de86569158caa5"}
Oct 30 20:59:50.213496 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:50.213447    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rcqw6" podStartSLOduration=3.825199102 podStartE2EDuration="12.213433144s" totalImagesPullingTime="8.388234042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:50.213020195 +0000 UTC m=+12.777909092" watchObservedRunningTime="2026-10-30 20:59:50.213433144 +0000 UTC m=+12.778322042"
Oct 30 20:59:51.180954 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:51.180920    2567 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="d6900617e0e0dc622f3ba39e3ea4331cd10f6f64dc1ed0a5afc20b16cccd6a9f" exitCode=0
Oct 30 20:59:51.181394 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:51.181004    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerDied","Data":"d6900617e0e0dc622f3ba39e3ea4331cd10f6f64dc1ed0a5afc20b16cccd6a9f"}
Oct 30 20:59:51.183047 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:51.183026    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qlqw5" event={"ID":"b0cf3a14-78d5-4493-a209-25a4f0233d59","Type":"ContainerStarted","Data":"a5a23daeaf5b346a3e371a1c17607a04182e9d266e07726d6da6ec9af4c3f81e"}
Oct 30 20:59:51.961959 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:51.961915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:51.962149 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:51.961915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:51.962149 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:51.962060    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:51.962276 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:51.962190    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:51.962276 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:51.961924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:51.962373 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:51.962300    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:53.190149 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:53.190112    2567 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="22bcdbef6544e99519ec0db5864652411f8564e52c854a8b1f2901bc35659934" exitCode=0
Oct 30 20:59:53.190954 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:53.190171    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerDied","Data":"22bcdbef6544e99519ec0db5864652411f8564e52c854a8b1f2901bc35659934"}
Oct 30 20:59:53.265289 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:53.265246    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qlqw5" podStartSLOduration=5.056346645 podStartE2EDuration="15.265233797s" totalImagesPullingTime="10.208887152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:51.291604214 +0000 UTC m=+13.856493116" watchObservedRunningTime="2026-10-30 20:59:53.265233797 +0000 UTC m=+15.830122698"
Oct 30 20:59:53.962416 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:53.962358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:53.962599 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:53.962357    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:53.962599 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:53.962498    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:53.962599 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:53.962515    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:53.962773 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:53.962595    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:53.962773 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:53.962674    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:54.196321 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:54.196280    2567 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="72471c4421638e2ed89328a620b293b487558312cd99d0ccc0c33f77427bda2d" exitCode=0
Oct 30 20:59:54.196813 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:54.196329    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerDied","Data":"72471c4421638e2ed89328a620b293b487558312cd99d0ccc0c33f77427bda2d"}
Oct 30 20:59:55.203158 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.202994    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" event={"ID":"458e6a65-37ba-42b3-a21a-a411d302093c","Type":"ContainerStarted","Data":"db7a31c8add3ca77d443ea038abca3d6e8ba54002d1cbfe22c8102c060e0033b"}
Oct 30 20:59:55.205109 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.205089    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 20:59:55.205407 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.205388    2567 generic.go:356] "Generic (PLEG): container finished" podID="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" containerID="dbff679efa30f7d832f9c1178227c8a770b06968488154e1f5542fa95010613a" exitCode=1
Oct 30 20:59:55.205467 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.205429    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"11a01e79de60011c972c995a672603c9087b3b407ac6f0a73242900195a8eed9"}
Oct 30 20:59:55.205500 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.205480    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"5bcfa820637bdc792a8daa4f3952849bd3533e256be6cc74f108ca68ca304011"}
Oct 30 20:59:55.205534 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.205509    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerDied","Data":"dbff679efa30f7d832f9c1178227c8a770b06968488154e1f5542fa95010613a"}
Oct 30 20:59:55.205562 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.205538    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"0af46a4b945a633d6d662378e20b94cebab2486c727b46e32bc1f644f4c01c9c"}
Oct 30 20:59:55.273637 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.273586    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mmfcw" podStartSLOduration=7.502930653 podStartE2EDuration="17.273567441s" totalImagesPullingTime="9.770636788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:55.259454978 +0000 UTC m=+17.824343876" watchObservedRunningTime="2026-10-30 20:59:55.273567441 +0000 UTC m=+17.838456340"
Oct 30 20:59:55.662183 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.661907    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:55.662183 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.662154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:55.662350 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.662061    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:55.662350 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.662243    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:55.662350 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.662265    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret podName:4c1d2b37-38ac-4aca-90d3-144d6537006f nodeName:}" failed. No retries permitted until 2026-10-30 21:00:11.662247017 +0000 UTC m=+34.227135894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret") pod "global-pull-secret-syncer-cpdvw" (UID: "4c1d2b37-38ac-4aca-90d3-144d6537006f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:59:55.662350 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.662282    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:11.662273176 +0000 UTC m=+34.227162051 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:59:55.763440 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.763405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:55.763588 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.763571    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:59:55.763643 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.763590    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:59:55.763643 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.763601    2567 projected.go:196] Error preparing data for projected volume kube-api-access-pq4x6 for pod openshift-network-diagnostics/network-check-target-hkm5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:55.763706 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.763648    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6 podName:09b454b3-3ffa-4639-9fb9-5fbc06ebd0da nodeName:}" failed. No retries permitted until 2026-10-30 21:00:11.763635504 +0000 UTC m=+34.328524407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pq4x6" (UniqueName: "kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6") pod "network-check-target-hkm5p" (UID: "09b454b3-3ffa-4639-9fb9-5fbc06ebd0da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:59:55.962115 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.962019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:55.962115 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.962058    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:55.962115 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:55.962022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:55.962363 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.962127    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:55.962363 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.962200    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:55.962363 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:55.962273    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:56.210211 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:56.210184    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 20:59:56.210586 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:56.210561    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"d8b01ca4e73c27ccfb7e53d9d2f09de0d5c57d3f97bd5655fc226ffcacc73767"}
Oct 30 20:59:56.210644 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:56.210637    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"9538964129fde7ed150cbb0f07706c52ae37b0e74d87f1aca929b6eae017bbda"}
Oct 30 20:59:57.962985 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:57.962945    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:57.963420 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:57.963031    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 20:59:57.963420 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:57.963050    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:57.963420 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:57.963123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:57.963420 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:57.963202    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:57.963420 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:57.963260    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:58.217440 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:58.217365    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 20:59:58.217722 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:58.217701    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"f12cafb7a892c49e0d357700cd1a84b60a9b18a434ba8060f1f762cdff190763"}
Oct 30 20:59:59.962227 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:59.962196    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 20:59:59.962227 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:59.962228    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 20:59:59.963310 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:59.962292    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 20:59:59.963310 ip-10-0-140-179 kubenswrapper[2567]: I1030 20:59:59.962311    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 20:59:59.963310 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:59.962379    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 20:59:59.963310 ip-10-0-140-179 kubenswrapper[2567]: E1030 20:59:59.962441    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 21:00:00.225367 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:00.225284    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:00:00.225719 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:00.225694    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"8d349608bcc6c22fd0835a675dd9421f6c4b570438cf190121bcf0c0cb762dcf"}
Oct 30 21:00:00.225963 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:00.225946    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:00.226025 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:00.226011    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:00.226101 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:00.226087    2567 scope.go:122] "RemoveContainer" containerID="dbff679efa30f7d832f9c1178227c8a770b06968488154e1f5542fa95010613a"
Oct 30 21:00:00.243924 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:00.243903    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:01.232193 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.232166    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:00:01.232570 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.232457    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" event={"ID":"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa","Type":"ContainerStarted","Data":"7a327f0970671331e46932a1f555da7ecac480191d6fdce7fd02f9f5116903d8"}
Oct 30 21:00:01.232729 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.232705    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:01.249791 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.249766    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:01.320955 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.320745    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt" podStartSLOduration=8.690211137 podStartE2EDuration="23.320730278s" totalImagesPullingTime="14.630519141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:01.320479166 +0000 UTC m=+23.885368064" watchObservedRunningTime="2026-10-30 21:00:01.320730278 +0000 UTC m=+23.885619175"
Oct 30 21:00:01.561337 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.561230    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cpdvw"]
Oct 30 21:00:01.561484 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.561367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:01.561554 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:01.561475    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 21:00:01.564907 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.564875    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8pcqj"]
Oct 30 21:00:01.565020 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.564929    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkm5p"]
Oct 30 21:00:01.565020 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.564989    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:01.565094 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:01.565074    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 21:00:01.565156 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:01.565142    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:01.565220 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:01.565211    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 21:00:02.951340 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:02.951295    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:02.962276 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:02.962245    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:02.962421 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:02.962287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:02.962421 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:02.962397    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 21:00:02.962545 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:02.962503    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 21:00:03.119264 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:03.119224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:03.962161 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:03.962081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:03.962526 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:03.962234    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 21:00:04.785398 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:04.785315    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:04.904689 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:04.904654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:04.961906 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:04.961872    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:04.961906 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:04.961904    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:04.962093 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:04.961985    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 21:00:04.962093 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:04.962051    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 21:00:05.476722 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:05.476687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:05.613805 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:05.613774    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:05.962769 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:05.962734    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:05.963008 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:05.962855    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 21:00:06.962531 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:06.962500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:06.963052 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:06.962505    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:06.963052 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:06.962632    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8pcqj" podUID="6ef41467-164c-4bc7-8870-d44ec9c655a9"
Oct 30 21:00:06.963052 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:06.962755    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkm5p" podUID="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da"
Oct 30 21:00:07.963245 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:07.963205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:07.963749 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:07.963315    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cpdvw" podUID="4c1d2b37-38ac-4aca-90d3-144d6537006f"
Oct 30 21:00:08.182307 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.182227    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeReady"
Oct 30 21:00:08.182466 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.182413    2567 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:00:08.236595 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.236563    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h4cm4"]
Oct 30 21:00:08.262449 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.262348    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gvgtv"]
Oct 30 21:00:08.262981 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.262957    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.278443 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.278421    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h4cm4"]
Oct 30 21:00:08.278557 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.278469    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gvgtv"]
Oct 30 21:00:08.278557 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.278550    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.279512 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.279483    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:00:08.279622 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.279534    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:00:08.279699 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.279688    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5jdx7\""
Oct 30 21:00:08.294592 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.294565    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:00:08.294741 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.294618    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:00:08.294741 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.294667    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:00:08.294741 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.294729    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-xstbj\""
Oct 30 21:00:08.353545 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.353505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrcms\" (UniqueName: \"kubernetes.io/projected/e5a2f410-97fd-4998-962c-dd2f8cd3281d-kube-api-access-vrcms\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.353545 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.353546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcpkx\" (UniqueName: \"kubernetes.io/projected/c645ee22-4c7d-443e-ae3a-0d50999f7c05-kube-api-access-xcpkx\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.353741 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.353569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.353741 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.353626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c645ee22-4c7d-443e-ae3a-0d50999f7c05-cert\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.353741 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.353714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a2f410-97fd-4998-962c-dd2f8cd3281d-tmp-dir\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.353856 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.353782    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5a2f410-97fd-4998-962c-dd2f8cd3281d-config-volume\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.454969 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.454882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.454969 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.454919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c645ee22-4c7d-443e-ae3a-0d50999f7c05-cert\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.454969 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.454974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a2f410-97fd-4998-962c-dd2f8cd3281d-tmp-dir\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.455216 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.455004    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5a2f410-97fd-4998-962c-dd2f8cd3281d-config-volume\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.455216 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.455035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrcms\" (UniqueName: \"kubernetes.io/projected/e5a2f410-97fd-4998-962c-dd2f8cd3281d-kube-api-access-vrcms\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.455216 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.455051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcpkx\" (UniqueName: \"kubernetes.io/projected/c645ee22-4c7d-443e-ae3a-0d50999f7c05-kube-api-access-xcpkx\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.455216 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.455050    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:00:08.455216 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.455131    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls podName:e5a2f410-97fd-4998-962c-dd2f8cd3281d nodeName:}" failed. No retries permitted until 2026-10-30 21:00:08.95511162 +0000 UTC m=+31.520000516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls") pod "dns-default-h4cm4" (UID: "e5a2f410-97fd-4998-962c-dd2f8cd3281d") : secret "dns-default-metrics-tls" not found
Oct 30 21:00:08.455471 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.455364    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a2f410-97fd-4998-962c-dd2f8cd3281d-tmp-dir\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.455673 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.455653    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5a2f410-97fd-4998-962c-dd2f8cd3281d-config-volume\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.459046 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.459026    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c645ee22-4c7d-443e-ae3a-0d50999f7c05-cert\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.477603 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.477561    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrcms\" (UniqueName: \"kubernetes.io/projected/e5a2f410-97fd-4998-962c-dd2f8cd3281d-kube-api-access-vrcms\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.477735 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.477668    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcpkx\" (UniqueName: \"kubernetes.io/projected/c645ee22-4c7d-443e-ae3a-0d50999f7c05-kube-api-access-xcpkx\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.615083 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.615044    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:08.653292 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.653264    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5blgd"]
Oct 30 21:00:08.679347 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.679118    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5blgd"]
Oct 30 21:00:08.679509 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.679234    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.699485 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.699459    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:00:08.699661 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.699645    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l57v4\""
Oct 30 21:00:08.700178 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.700160    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:00:08.700178 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.700169    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:00:08.700373 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.700358    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:00:08.738081 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.738038    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-8db7s-5kjq9"]
Oct 30 21:00:08.756799 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.755408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.757066 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.756997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c151da3-d2a5-46fe-97bc-51d030c13470-data-volume\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.757066 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.757038    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.757181 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.757096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c151da3-d2a5-46fe-97bc-51d030c13470-crio-socket\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.757181 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.757113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.757181 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.757162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcrlc\" (UniqueName: \"kubernetes.io/projected/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-api-access-rcrlc\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.759123 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.759087    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-8db7s-5kjq9"]
Oct 30 21:00:08.780794 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.780759    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:08.799552 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.799514    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:00:08.799749 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.799664    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bkjxf\""
Oct 30 21:00:08.814997 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.814972    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:00:08.857681 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.857889 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c151da3-d2a5-46fe-97bc-51d030c13470-crio-socket\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.857889 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857750    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.857889 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.857889 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.858073 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.857917    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:08.858073 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcrlc\" (UniqueName: \"kubernetes.io/projected/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-api-access-rcrlc\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.858073 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.857988    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:09.35796937 +0000 UTC m=+31.922858247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:08.858073 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.857987    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c151da3-d2a5-46fe-97bc-51d030c13470-crio-socket\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.858073 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.858041    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c151da3-d2a5-46fe-97bc-51d030c13470-data-volume\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.858280 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.858074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.858280 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.858108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.858280 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.858128    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:00:08.858280 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.858183    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:09.358169635 +0000 UTC m=+31.923058561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : configmap "kube-rbac-proxy" not found
Oct 30 21:00:08.858629 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.858610    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c151da3-d2a5-46fe-97bc-51d030c13470-data-volume\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.881923 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.881894    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcrlc\" (UniqueName: \"kubernetes.io/projected/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-api-access-rcrlc\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:08.904212 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.904174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:08.959360 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.959324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.959541 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.959382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:08.959541 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.959419    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.959541 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.959440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.959704 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.959569    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:00:08.959704 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.959652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.959704 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:08.959663    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls podName:e5a2f410-97fd-4998-962c-dd2f8cd3281d nodeName:}" failed. No retries permitted until 2026-10-30 21:00:09.959639056 +0000 UTC m=+32.524527946 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls") pod "dns-default-h4cm4" (UID: "e5a2f410-97fd-4998-962c-dd2f8cd3281d") : secret "dns-default-metrics-tls" not found
Oct 30 21:00:08.960087 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.960071    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.960185 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.960166    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.960706 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.960689    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.962358 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.962337    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:08.962527 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.962507    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:08.982513 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.982487    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:08.985535 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.985260    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fz2bf\""
Oct 30 21:00:08.985714 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.985693    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:00:08.985783 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.985716    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jljmv\""
Oct 30 21:00:08.986301 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.986282    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gvgtv"]
Oct 30 21:00:08.987461 ip-10-0-140-179 kubenswrapper[2567]: W1030 21:00:08.987413    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc645ee22_4c7d_443e_ae3a_0d50999f7c05.slice/crio-9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58 WatchSource:0}: Error finding container 9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58: Status 404 returned error can't find the container with id 9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58
Oct 30 21:00:08.999860 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.999815    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:00:08.999860 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:08.999856    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:00:09.038281 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.038244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:09.076531 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.076362    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:09.077487 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.077468    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"]
Oct 30 21:00:09.097990 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.097964    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:09.101920 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.101889    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"]
Oct 30 21:00:09.116955 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.116924    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mttl5\""
Oct 30 21:00:09.118038 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.117998    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:00:09.118169 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.118069    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:00:09.161423 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.161390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x9zv\" (UniqueName: \"kubernetes.io/projected/a4e28884-7c4b-4a04-a21c-ba0214022cbe-kube-api-access-4x9zv\") pod \"migrator-66df5d496b-6zrdn\" (UID: \"a4e28884-7c4b-4a04-a21c-ba0214022cbe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:09.184425 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.184394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:09.254298 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.254219    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gvgtv" event={"ID":"c645ee22-4c7d-443e-ae3a-0d50999f7c05","Type":"ContainerStarted","Data":"9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58"}
Oct 30 21:00:09.262604 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.262581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9zv\" (UniqueName: \"kubernetes.io/projected/a4e28884-7c4b-4a04-a21c-ba0214022cbe-kube-api-access-4x9zv\") pod \"migrator-66df5d496b-6zrdn\" (UID: \"a4e28884-7c4b-4a04-a21c-ba0214022cbe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:09.287536 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.287512    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9zv\" (UniqueName: \"kubernetes.io/projected/a4e28884-7c4b-4a04-a21c-ba0214022cbe-kube-api-access-4x9zv\") pod \"migrator-66df5d496b-6zrdn\" (UID: \"a4e28884-7c4b-4a04-a21c-ba0214022cbe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:09.304061 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.304034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:09.363652 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.363619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:09.363797 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.363683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:09.363797 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:09.363762    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:00:09.363797 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:09.363767    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:09.363956 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:09.363814    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:10.363801161 +0000 UTC m=+32.928690038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : configmap "kube-rbac-proxy" not found
Oct 30 21:00:09.363956 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:09.363860    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:10.363820433 +0000 UTC m=+32.928709308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:09.391863 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.391818    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-8db7s-5kjq9"]
Oct 30 21:00:09.392701 ip-10-0-140-179 kubenswrapper[2567]: W1030 21:00:09.392678    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc6f18b_e6e7_4909_8321_f7d14cf54ead.slice/crio-5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a WatchSource:0}: Error finding container 5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a: Status 404 returned error can't find the container with id 5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a
Oct 30 21:00:09.426238 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.426209    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:09.507295 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.507268    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wrhzf"]
Oct 30 21:00:09.523522 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.523444    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.529734 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.529711    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wrhzf"]
Oct 30 21:00:09.540435 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.540410    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 21:00:09.540435 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.540410    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 21:00:09.540608 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.540443    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 21:00:09.540608 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.540576    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 21:00:09.540608 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.540593    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m9rs2\""
Oct 30 21:00:09.667061 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.667021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6sr\" (UniqueName: \"kubernetes.io/projected/aac5c587-aab5-48db-9fc7-adc2f648ef45-kube-api-access-tx6sr\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.667235 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.667155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-cabundle\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.667235 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.667192    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-key\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.748483 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.748441    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"]
Oct 30 21:00:09.750004 ip-10-0-140-179 kubenswrapper[2567]: W1030 21:00:09.749970    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e28884_7c4b_4a04_a21c_ba0214022cbe.slice/crio-ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62 WatchSource:0}: Error finding container ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62: Status 404 returned error can't find the container with id ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62
Oct 30 21:00:09.767874 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.767795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6sr\" (UniqueName: \"kubernetes.io/projected/aac5c587-aab5-48db-9fc7-adc2f648ef45-kube-api-access-tx6sr\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.767981 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.767875    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-cabundle\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.767981 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.767900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-key\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.768609 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.768585    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-cabundle\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.770963 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.770939    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-key\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.789983 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.789951    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6sr\" (UniqueName: \"kubernetes.io/projected/aac5c587-aab5-48db-9fc7-adc2f648ef45-kube-api-access-tx6sr\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.852003 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.851951    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:09.923030 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.922996    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:09.964334 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.964303    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:09.969941 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.969915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:09.970089 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:09.970066    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:00:09.970159 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:09.970123    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls podName:e5a2f410-97fd-4998-962c-dd2f8cd3281d nodeName:}" failed. No retries permitted until 2026-10-30 21:00:11.970105232 +0000 UTC m=+34.534994109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls") pod "dns-default-h4cm4" (UID: "e5a2f410-97fd-4998-962c-dd2f8cd3281d") : secret "dns-default-metrics-tls" not found
Oct 30 21:00:09.982203 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.982177    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:00:09.982337 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:09.982238    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-56hzc\""
Oct 30 21:00:10.158818 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.158792    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wrhzf"]
Oct 30 21:00:10.160041 ip-10-0-140-179 kubenswrapper[2567]: W1030 21:00:10.159999    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac5c587_aab5_48db_9fc7_adc2f648ef45.slice/crio-faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6 WatchSource:0}: Error finding container faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6: Status 404 returned error can't find the container with id faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6
Oct 30 21:00:10.258430 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.258386    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf" event={"ID":"aac5c587-aab5-48db-9fc7-adc2f648ef45","Type":"ContainerStarted","Data":"faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6"}
Oct 30 21:00:10.260218 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.260151    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8db7s-5kjq9" event={"ID":"4bc6f18b-e6e7-4909-8321-f7d14cf54ead","Type":"ContainerStarted","Data":"5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a"}
Oct 30 21:00:10.261962 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.261907    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn" event={"ID":"a4e28884-7c4b-4a04-a21c-ba0214022cbe","Type":"ContainerStarted","Data":"ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62"}
Oct 30 21:00:10.367737 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.367616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:10.373052 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.373011    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:10.373215 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:10.373108    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:10.373215 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:10.373148    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:00:10.373215 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:10.373214    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:12.373195071 +0000 UTC m=+34.938083953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : configmap "kube-rbac-proxy" not found
Oct 30 21:00:10.373375 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:10.373230    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:10.373375 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:10.373281    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:12.373266089 +0000 UTC m=+34.938154979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:11.267301 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.267262    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn" event={"ID":"a4e28884-7c4b-4a04-a21c-ba0214022cbe","Type":"ContainerStarted","Data":"4f7669a540d9e89564271e17f17395bc9fecc4a6f07e46210d24bd6d1e78d090"}
Oct 30 21:00:11.309208 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.309096    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn" podStartSLOduration=1.015300429 podStartE2EDuration="2.309081295s" totalImagesPullingTime="1.293780866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:11.308988283 +0000 UTC m=+33.873877200" watchObservedRunningTime="2026-10-30 21:00:11.309081295 +0000 UTC m=+33.873970186"
Oct 30 21:00:11.685670 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.685624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:11.685877 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.685691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:11.685877 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:11.685817    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:00:11.685999 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:11.685898    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:43.685879446 +0000 UTC m=+66.250768334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : secret "metrics-daemon-secret" not found
Oct 30 21:00:11.692855 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.690880    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:11.777505 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.777478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:11.786945 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.786818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:11.790179 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.790134    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:11.989379 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:11.989340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:11.989622 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:11.989598    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:00:11.989700 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:11.989670    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls podName:e5a2f410-97fd-4998-962c-dd2f8cd3281d nodeName:}" failed. No retries permitted until 2026-10-30 21:00:15.98965126 +0000 UTC m=+38.554540161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls") pod "dns-default-h4cm4" (UID: "e5a2f410-97fd-4998-962c-dd2f8cd3281d") : secret "dns-default-metrics-tls" not found
Oct 30 21:00:12.015476 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.015231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:12.088352 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.088327    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cpdvw"]
Oct 30 21:00:12.093415 ip-10-0-140-179 kubenswrapper[2567]: W1030 21:00:12.093383    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1d2b37_38ac_4aca_90d3_144d6537006f.slice/crio-c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc WatchSource:0}: Error finding container c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc: Status 404 returned error can't find the container with id c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc
Oct 30 21:00:12.272067 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.272033    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cpdvw" event={"ID":"4c1d2b37-38ac-4aca-90d3-144d6537006f","Type":"ContainerStarted","Data":"c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc"}
Oct 30 21:00:12.274759 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.274730    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8db7s-5kjq9" event={"ID":"4bc6f18b-e6e7-4909-8321-f7d14cf54ead","Type":"ContainerStarted","Data":"5d6b5e91782bc41d9b74e7477ba95447c2c9ca87870109c233fca560b50e7ff6"}
Oct 30 21:00:12.318205 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.318179    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkm5p"]
Oct 30 21:00:12.321121 ip-10-0-140-179 kubenswrapper[2567]: W1030 21:00:12.320951    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b454b3_3ffa_4639_9fb9_5fbc06ebd0da.slice/crio-2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15 WatchSource:0}: Error finding container 2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15: Status 404 returned error can't find the container with id 2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15
Oct 30 21:00:12.331580 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.331534    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-8db7s-5kjq9" podStartSLOduration=1.954899577 podStartE2EDuration="4.33151867s" totalImagesPullingTime="2.376619093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:12.320584816 +0000 UTC m=+34.885473731" watchObservedRunningTime="2026-10-30 21:00:12.33151867 +0000 UTC m=+34.896407599"
Oct 30 21:00:12.392662 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.392620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:12.392879 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:12.392738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:12.393280 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:12.393260    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:00:12.393365 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:12.393325    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:16.393305676 +0000 UTC m=+38.958194567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : configmap "kube-rbac-proxy" not found
Oct 30 21:00:12.393981 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:12.393958    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:12.394102 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:12.394018    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:16.394002012 +0000 UTC m=+38.958890901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:13.278709 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.278657    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gvgtv" event={"ID":"c645ee22-4c7d-443e-ae3a-0d50999f7c05","Type":"ContainerStarted","Data":"e2e83889c28adb6086b68d4169d4a01bbf3ad38d9e57bb7b85aa409d71569706"}
Oct 30 21:00:13.281460 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.281055    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf" event={"ID":"aac5c587-aab5-48db-9fc7-adc2f648ef45","Type":"ContainerStarted","Data":"ccf6cbddd545cfb3520ebe66b2bc0a924949b93ca886b7a12359ed44aa83147f"}
Oct 30 21:00:13.282419 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.282392    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkm5p" event={"ID":"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da","Type":"ContainerStarted","Data":"2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15"}
Oct 30 21:00:13.370754 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.370696    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gvgtv" podStartSLOduration=2.106644708 podStartE2EDuration="5.370682015s" totalImagesPullingTime="3.264037307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:13.327061522 +0000 UTC m=+35.891950440" watchObservedRunningTime="2026-10-30 21:00:13.370682015 +0000 UTC m=+35.935570890"
Oct 30 21:00:13.370969 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.370879    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf" podStartSLOduration=1.909364844 podStartE2EDuration="4.370871554s" totalImagesPullingTime="2.46150671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:13.369908804 +0000 UTC m=+35.934797703" watchObservedRunningTime="2026-10-30 21:00:13.370871554 +0000 UTC m=+35.935760452"
Oct 30 21:00:13.607688 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.607639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:13.752318 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.752288    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:13.890364 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:13.890286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:14.093030 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:14.092996    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:14.250473 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:14.250384    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:14.409441 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:14.409405    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:14.531790 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:14.531609    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:14.596150 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:14.596122    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wsmb_3b759b7b-3f86-427d-a1f7-6382fbe96464/dns-node-resolver/0.log"
Oct 30 21:00:15.116901 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.116865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:15.244194 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.244157    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:15.362173 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.362075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:15.483602 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.483570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:15.597384 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.597358    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lb6cv_b346fb18-266d-44b4-998e-afdaf5ae0085/node-ca/0.log"
Oct 30 21:00:15.681176 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.681141    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:15.945636 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:15.945493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.027973 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.027935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:16.028153 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:16.028100    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:00:16.028200 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:16.028166    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls podName:e5a2f410-97fd-4998-962c-dd2f8cd3281d nodeName:}" failed. No retries permitted until 2026-10-30 21:00:24.028145645 +0000 UTC m=+46.593034521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls") pod "dns-default-h4cm4" (UID: "e5a2f410-97fd-4998-962c-dd2f8cd3281d") : secret "dns-default-metrics-tls" not found
Oct 30 21:00:16.066694 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.066649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.196474 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.196396    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gvgtv_c645ee22-4c7d-443e-ae3a-0d50999f7c05/serve-healthcheck-canary/0.log"
Oct 30 21:00:16.203113 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.203083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.293813 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.293771    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkm5p" event={"ID":"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da","Type":"ContainerStarted","Data":"ec8354d8b34bc5ae87d320f1fb7bf939c7f5e55f241d0b32fabbaa97f8b176d2"}
Oct 30 21:00:16.294042 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.293956    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:16.320349 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.320302    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.337395 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.337332    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hkm5p" podStartSLOduration=35.088445498 podStartE2EDuration="38.337315911s" totalImagesPullingTime="3.248870413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:16.337260085 +0000 UTC m=+38.902148985" watchObservedRunningTime="2026-10-30 21:00:16.337315911 +0000 UTC m=+38.902204812"
Oct 30 21:00:16.431401 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.431358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:16.431590 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.431444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:16.431590 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:16.431560    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:16.431697 ip-10-0-140-179 kubenswrapper[2567]: E1030 21:00:16.431623    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:24.431605366 +0000 UTC m=+46.996494259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:16.432065 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.432046    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:16.798521 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.798488    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-6zrdn_a4e28884-7c4b-4a04-a21c-ba0214022cbe/migrator/0.log"
Oct 30 21:00:16.848402 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.848363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.975631 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:16.975585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:17.173119 ip-10-0-140-179 kubenswrapper[2567]: I1030 21:00:17.173039    2567 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:00:17.173194 ip-10-0-140-179 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:00:17.183257 ip-10-0-140-179 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:00:17.183459 ip-10-0-140-179 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:00:17.183515 ip-10-0-140-179 systemd[1]: kubelet.service: Consumed 2.572s CPU time.
Oct 30 21:00:17.192576 ip-10-0-140-179 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:00:17.260352 ip-10-0-140-179 kubenswrapper[4549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:00:17.260352 ip-10-0-140-179 kubenswrapper[4549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:00:17.260352 ip-10-0-140-179 kubenswrapper[4549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:00:17.260352 ip-10-0-140-179 kubenswrapper[4549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:00:17.262559 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262468    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.262559 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262560    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.262559 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262563    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262570    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262575    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262578    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262581    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262584    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262592    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262595    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262597    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262600    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262602    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262606    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262609    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262611    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262614    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262616    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262619    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262622    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262624    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.262652 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262627    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262629    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262632    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262635    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262638    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262640    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262643    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262645    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262648    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262650    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262653    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262656    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262658    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262661    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262663    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262666    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262670    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262683    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262697    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262700    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.263170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262703    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262705    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262708    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262710    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262713    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262715    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262721    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262725    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262727    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262730    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262733    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262735    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262738    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262740    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262743    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262745    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262748    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262751    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262753    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262755    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.263649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262758    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262760    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262762    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262765    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262767    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262770    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262772    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262774    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262777    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262779    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262782    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262790    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262793    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262796    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262799    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262801    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262804    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262806    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262810    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262814    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.264151 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262816    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262819    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262821    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262847    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262852    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262856    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262859    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262862    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262865    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262867    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262870    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262873    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262875    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262878    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262881    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262883    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262885    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262888    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262890    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.264636 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262893    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262897    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262899    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262902    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262904    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.262913    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263774    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263782    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263785    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263788    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263793    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263799    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263802    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263805    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263808    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263810    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263813    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263816    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263818    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.265125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263821    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263823    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263843    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263845    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263848    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263851    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263854    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263856    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263859    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263861    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263864    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263866    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263869    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263872    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263875    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263877    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263880    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263882    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263885    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263888    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.265600 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263891    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263894    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263896    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263899    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263901    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263904    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263907    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263909    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263912    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263914    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263917    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263919    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263922    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263924    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263927    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263930    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263932    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263934    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263937    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263939    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.266150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263942    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263944    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263948    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263950    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263953    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263955    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263957    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263960    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263963    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263966    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263968    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263971    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263973    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263976    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263979    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263981    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263984    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263986    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263989    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263991    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.266688 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263994    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263996    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.263998    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264001    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264003    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264005    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264008    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264010    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264013    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264015    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264018    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264020    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264022    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264025    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264028    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264030    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264033    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264035    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264038    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.267243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264040    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264043    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264047    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264051    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264054    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264057    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264060    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264063    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264065    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264067    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264072    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264075    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264082    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264085    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264639    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264645    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264648    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264651    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264653    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.267719 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264655    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264658    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264661    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264663    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264666    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264668    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264671    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264674    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264677    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264680    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264682    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264685    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264687    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264690    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264693    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264695    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264698    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264700    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264703    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264706    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.268179 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264709    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264711    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264714    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264719    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264721    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264724    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264726    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264728    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264733    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264737    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264740    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264743    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264745    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264748    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264750    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264753    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264756    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264758    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.268682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264761    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264764    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264766    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264769    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264771    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264774    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264776    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264778    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264781    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264783    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264786    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264788    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264791    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264793    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264797    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264799    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264802    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264806    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264808    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264811    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.269236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264813    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264816    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264818    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264820    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264823    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264840    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264843    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264846    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264848    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264851    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264853    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264856    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264858    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264861    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264863    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264865    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264868    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264870    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264873    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264875    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.269728 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264877    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264880    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264883    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264885    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264889    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264892    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264895    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264898    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264901    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264905    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264907    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264911    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264915    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264918    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264923    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264926    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264928    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264931    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264933    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.270265 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264936    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264938    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264941    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.264943    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265041    4549 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265056    4549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265063    4549 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265068    4549 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265073    4549 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265076    4549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265081    4549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265085    4549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265088    4549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265091    4549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265095    4549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265098    4549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265101    4549 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265106    4549 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265109    4549 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265112    4549 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265115    4549 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265119    4549 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265125    4549 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:00:17.271073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265128    4549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265131    4549 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265134    4549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265138    4549 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265142    4549 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265144    4549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265147    4549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265151    4549 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265154    4549 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265157    4549 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265160    4549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265163    4549 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265166    4549 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265170    4549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265173    4549 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265176    4549 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265179    4549 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265182    4549 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265185    4549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265192    4549 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265195    4549 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265198    4549 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265201    4549 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265204    4549 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:00:17.271795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265209    4549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265211    4549 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265217    4549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265220    4549 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265224    4549 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265227    4549 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265230    4549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265234    4549 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265238    4549 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265241    4549 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265244    4549 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265248    4549 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265251    4549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265254    4549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265257    4549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265260    4549 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265263    4549 flags.go:64] FLAG: --help="false"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265266    4549 flags.go:64] FLAG: --hostname-override="ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265269    4549 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265272    4549 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265275    4549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265278    4549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265282    4549 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265284    4549 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:00:17.272510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265287    4549 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265290    4549 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265293    4549 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265296    4549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265298    4549 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265301    4549 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265304    4549 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265307    4549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265310    4549 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265313    4549 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265317    4549 flags.go:64] FLAG: --lock-file=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265320    4549 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265323    4549 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265326    4549 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265332    4549 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265336    4549 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265340    4549 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265343    4549 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265346    4549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265349    4549 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265352    4549 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265355    4549 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265359    4549 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265363    4549 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265366    4549 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:00:17.273109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265369    4549 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265372    4549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265375    4549 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265378    4549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265381    4549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265384    4549 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265387    4549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265394    4549 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265397    4549 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265400    4549 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265403    4549 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265406    4549 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265409    4549 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265411    4549 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265414    4549 flags.go:64] FLAG: --port="10250"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265417    4549 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265420    4549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b30b1ea0f871ada"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265424    4549 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265427    4549 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265430    4549 flags.go:64] FLAG: --register-node="true"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265433    4549 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265437    4549 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265441    4549 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265444    4549 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265448    4549 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265451    4549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:00:17.273716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265454    4549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265458    4549 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265461    4549 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265464    4549 flags.go:64] FLAG: --runonce="false"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265467    4549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265470    4549 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265473    4549 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265476    4549 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265478    4549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265482    4549 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265485    4549 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265488    4549 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265490    4549 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265493    4549 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265496    4549 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265499    4549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265502    4549 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265505    4549 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265507    4549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265513    4549 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265516    4549 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265519    4549 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265522    4549 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265527    4549 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265530    4549 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:00:17.274356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265533    4549 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265536    4549 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265539    4549 flags.go:64] FLAG: --v="2"
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265545    4549 flags.go:64] FLAG: --version="false"
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265550    4549 flags.go:64] FLAG: --vmodule=""
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265554    4549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.265558    4549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265718    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265722    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265725    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265728    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265731    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265733    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265736    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265739    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265742    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265744    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265747    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265750    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265752    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265755    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.274984 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265757    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265760    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265762    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265765    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265767    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265770    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265772    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265775    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265777    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265781    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265784    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265786    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265789    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265791    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265796    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265798    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265801    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265803    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265806    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265809    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.275490 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265811    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265814    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265816    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265819    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265821    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265838    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265841    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265844    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265846    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265850    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265854    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265858    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265860    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265864    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265866    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265869    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265871    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265874    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265876    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265879    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.275987 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265881    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265886    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265889    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265891    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265894    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265896    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265901    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265904    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265907    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265909    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265912    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265915    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265918    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265921    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265923    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265926    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265928    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265931    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265933    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.276512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265936    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265938    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265941    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265943    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265946    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265948    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265951    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265953    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265956    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265958    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265961    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265963    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265966    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265968    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265972    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265975    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265979    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265982    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265984    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.277173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265988    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265991    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265993    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265996    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.265998    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266001    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266003    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266006    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266009    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266011    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266014    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266016    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266019    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266021    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.266048    4549 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266083    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.277675 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266088    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266091    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266093    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266095    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266098    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266101    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266103    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266106    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266108    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266111    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266114    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266116    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266120    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266123    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266126    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266129    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266134    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266136    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266139    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.278505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266141    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266144    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266147    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266149    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266152    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266155    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266159    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266161    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266164    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266166    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266169    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266171    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266173    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266176    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266179    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266181    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266184    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266186    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266189    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266191    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.279250 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266194    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266197    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266199    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266202    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266204    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266208    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266210    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266213    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266215    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266219    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266222    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266224    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266227    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266229    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266232    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266234    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266237    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266240    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266243    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266245    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.279924 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266248    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266250    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266253    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266256    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266258    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266260    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266263    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266265    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266268    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266270    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266273    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266275    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266278    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266280    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266283    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266285    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266289    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266294    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266297    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266300    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.280457 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266303    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266306    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266309    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266311    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266314    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266317    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266319    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266322    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266324    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266327    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266330    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266332    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266335    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266337    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266340    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266342    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266345    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266347    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266350    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.280970 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266353    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266355    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266358    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266360    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266363    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266365    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.266368    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.270901    4549 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.270927    4549 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.270953    4549 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.270960    4549 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271154    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271162    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271167    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271172    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271176    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271181    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271185    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271197    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271202    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.281443 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271206    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271211    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271215    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271219    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271223    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271228    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271232    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271236    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271241    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271245    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271249    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271260    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271265    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271269    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271273    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271277    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271281    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271285    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271289    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.281945 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271293    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271300    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271304    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271309    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271313    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271322    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271326    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271331    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271335    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271340    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271344    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271348    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271352    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271357    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271361    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271365    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271369    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271377    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271382    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271386    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.282400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271390    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271397    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271403    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271409    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271414    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271418    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271422    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271427    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271431    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271435    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271444    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271449    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271453    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271461    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271468    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271473    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271477    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271482    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271487    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.283026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271491    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271496    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271501    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271505    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271510    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271514    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271519    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271528    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271532    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271684    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271711    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271716    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271720    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271725    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271731    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271737    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271742    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271746    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271750    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.283475 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271752    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271755    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271758    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271760    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271763    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271765    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271768    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271771    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271774    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271778    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271780    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271783    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271786    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271788    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271791    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271794    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271797    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271801    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271803    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.283957 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.271806    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.271877    4549 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272032    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272038    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272041    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272044    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272047    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272050    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272052    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272055    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272058    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272061    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272064    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272066    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272069    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272072    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.284501 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272074    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272077    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272079    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272082    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272084    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272087    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272089    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272091    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272094    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272096    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272100    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272102    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272105    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272108    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272111    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272113    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272116    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272118    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272121    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.284910 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272123    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272126    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272128    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272131    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272134    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272138    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272140    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272143    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272145    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272148    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272150    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272153    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272155    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272158    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272160    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272163    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272165    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272168    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272171    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272173    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.285368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272177    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272182    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272186    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272190    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272194    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272198    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272201    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272204    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272207    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272210    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272212    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272216    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272218    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272221    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272223    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272226    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272228    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272231    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272234    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.285900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272237    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272239    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272242    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272244    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272247    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272249    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272252    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272254    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272257    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272259    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272262    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272264    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272267    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272269    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272272    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272275    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272278    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272280    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272283    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.286359 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272286    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272288    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272291    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272294    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272296    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272299    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272301    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272304    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272306    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272308    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272311    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272314    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272317    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272320    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272322    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.272348    4549 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:00:17.286810 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272380    4549 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272384    4549 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272387    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272390    4549 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272392    4549 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272395    4549 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272397    4549 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272400    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272402    4549 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272405    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272408    4549 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272410    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272413    4549 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272415    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272418    4549 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272420    4549 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272423    4549 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272426    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272429    4549 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:00:17.287211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272431    4549 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272434    4549 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272436    4549 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272439    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272441    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272443    4549 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272449    4549 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272452    4549 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272454    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272457    4549 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272459    4549 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272462    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272465    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272467    4549 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272470    4549 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272472    4549 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272475    4549 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272477    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272480    4549 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272483    4549 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:00:17.287669 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272485    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272487    4549 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272490    4549 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272493    4549 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272495    4549 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272497    4549 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272500    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272502    4549 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272505    4549 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272508    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272511    4549 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272513    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272516    4549 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272519    4549 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272521    4549 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272524    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272527    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272530    4549 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272535    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272538    4549 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:00:17.288164 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272540    4549 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272543    4549 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272545    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272548    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272551    4549 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272553    4549 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272555    4549 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272558    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272561    4549 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272563    4549 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272566    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272569    4549 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272571    4549 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272574    4549 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272577    4549 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272579    4549 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272583    4549 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272585    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272588    4549 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:00:17.288880 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272590    4549 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272593    4549 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272595    4549 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272598    4549 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272600    4549 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272603    4549 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272605    4549 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272608    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272610    4549 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272613    4549 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272615    4549 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272619    4549 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272622    4549 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272625    4549 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272629    4549 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272633    4549 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272637    4549 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272640    4549 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272643    4549 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:00:17.289597 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272646    4549 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272649    4549 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272651    4549 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272654    4549 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272657    4549 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272660    4549 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272663    4549 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272665    4549 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:17.272668    4549 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.272727    4549 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.273765    4549 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.273848    4549 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.274041    4549 server.go:1014] "Starting client certificate rotation"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.274176    4549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.274227    4549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 20:54:38 +0000 UTC" deadline="2028-05-24 13:40:16.384549864 +0000 UTC"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.274254    4549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13720h39m59.110297288s"
Oct 30 21:00:17.290255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.274501    4549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:00:17.290688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.275630    4549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:00:17.290688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.277178    4549 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:00:17.290688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.279215    4549 log.go:25] "Validated CRI v1 image API"
Oct 30 21:00:17.290688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.279932    4549 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:00:17.290688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.281532    4549 fs.go:135] Filesystem UUIDs: map[3a30494f-d12a-4d42-b5b1-b56af3b106e8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9650cb35-4222-49ff-b6d6-8a12efb2e6de:/dev/nvme0n1p3]
Oct 30 21:00:17.290858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.281553    4549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da/volumes/kubernetes.io~projected/kube-api-access-pq4x6:{mountpoint:/var/lib/kubelet/pods/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da/volumes/kubernetes.io~projected/kube-api-access-pq4x6 major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b759b7b-3f86-427d-a1f7-6382fbe96464/volumes/kubernetes.io~projected/kube-api-access-w4z66:{mountpoint:/var/lib/kubelet/pods/3b759b7b-3f86-427d-a1f7-6382fbe96464/volumes/kubernetes.io~projected/kube-api-access-w4z66 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c151da3-d2a5-46fe-97bc-51d030c13470/volumes/kubernetes.io~projected/kube-api-access-rcrlc:{mountpoint:/var/lib/kubelet/pods/3c151da3-d2a5-46fe-97bc-51d030c13470/volumes/kubernetes.io~projected/kube-api-access-rcrlc major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/458e6a65-37ba-42b3-a21a-a411d302093c/volumes/kubernetes.io~projected/kube-api-access-ftdkm:{mountpoint:/var/lib/kubelet/pods/458e6a65-37ba-42b3-a21a-a411d302093c/volumes/kubernetes.io~projected/kube-api-access-ftdkm major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bc6f18b-e6e7-4909-8321-f7d14cf54ead/volumes/kubernetes.io~projected/kube-api-access-bbjgn:{mountpoint:/var/lib/kubelet/pods/4bc6f18b-e6e7-4909-8321-f7d14cf54ead/volumes/kubernetes.io~projected/kube-api-access-bbjgn major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c1d2b37-38ac-4aca-90d3-144d6537006f/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/4c1d2b37-38ac-4aca-90d3-144d6537006f/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fc96670-a561-4f89-896a-a1b8369560a8/volumes/kubernetes.io~projected/kube-api-access-799cl:{mountpoint:/var/lib/kubelet/pods/4fc96670-a561-4f89-896a-a1b8369560a8/volumes/kubernetes.io~projected/kube-api-access-799cl major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22/volumes/kubernetes.io~projected/kube-api-access-bvdsw:{mountpoint:/var/lib/kubelet/pods/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22/volumes/kubernetes.io~projected/kube-api-access-bvdsw major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ef41467-164c-4bc7-8870-d44ec9c655a9/volumes/kubernetes.io~projected/kube-api-access-hnvzb:{mountpoint:/var/lib/kubelet/pods/6ef41467-164c-4bc7-8870-d44ec9c655a9/volumes/kubernetes.io~projected/kube-api-access-hnvzb major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81eb3986-da71-4e0e-862b-c84d3f33815c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/81eb3986-da71-4e0e-862b-c84d3f33815c/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4e28884-7c4b-4a04-a21c-ba0214022cbe/volumes/kubernetes.io~projected/kube-api-access-4x9zv:{mountpoint:/var/lib/kubelet/pods/a4e28884-7c4b-4a04-a21c-ba0214022cbe/volumes/kubernetes.io~projected/kube-api-access-4x9zv major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a7034506-9c4f-4252-8564-31d0bbe36a16/volumes/kubernetes.io~projected/kube-api-access-mqlkc:{mountpoint:/var/lib/kubelet/pods/a7034506-9c4f-4252-8564-31d0bbe36a16/volumes/kubernetes.io~projected/kube-api-access-mqlkc major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aac5c587-aab5-48db-9fc7-adc2f648ef45/volumes/kubernetes.io~projected/kube-api-access-tx6sr:{mountpoint:/var/lib/kubelet/pods/aac5c587-aab5-48db-9fc7-adc2f648ef45/volumes/kubernetes.io~projected/kube-api-access-tx6sr major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aac5c587-aab5-48db-9fc7-adc2f648ef45/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/aac5c587-aab5-48db-9fc7-adc2f648ef45/volumes/kubernetes.io~secret/signing-key major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~projected/kube-api-access-pxj6c:{mountpoint:/var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~projected/kube-api-access-pxj6c major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b346fb18-266d-44b4-998e-afdaf5ae0085/volumes/kubernetes.io~projected/kube-api-access-w5n8v:{mountpoint:/var/lib/kubelet/pods/b346fb18-266d-44b4-998e-afdaf5ae0085/volumes/kubernetes.io~projected/kube-api-access-w5n8v major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c645ee22-4c7d-443e-ae3a-0d50999f7c05/volumes/kubernetes.io~projected/kube-api-access-xcpkx:{mountpoint:/var/lib/kubelet/pods/c645ee22-4c7d-443e-ae3a-0d50999f7c05/volumes/kubernetes.io~projected/kube-api-access-xcpkx major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c645ee22-4c7d-443e-ae3a-0d50999f7c05/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/c645ee22-4c7d-443e-ae3a-0d50999f7c05/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volumes/kubernetes.io~projected/kube-api-access-56x7c:{mountpoint:/var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volumes/kubernetes.io~projected/kube-api-access-56x7c major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e5a2f410-97fd-4998-962c-dd2f8cd3281d/volumes/kubernetes.io~projected/kube-api-access-vrcms:{mountpoint:/var/lib/kubelet/pods/e5a2f410-97fd-4998-962c-dd2f8cd3281d/volumes/kubernetes.io~projected/kube-api-access-vrcms major:0 minor:197 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/5083d9f9dbf996d46d75d83905e5c54149dccd9f30a5baeffac9208fbd1bd869/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/250828c77b99af0253a1eff6e8718bbfbcf9e9ecf81d145c27e19f692ab0e592/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/92a911b9a7b68027f5488b8d4acd29ca6d8cc42b2c372aa3f5c2dc0fbe66ad63/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/c173557fd4a40d8321a69ec203a6f5e0a2adcebf926d7119b7e878bde202aa0a/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/8c7db7d6fa49a242219e480494443ab4954d598c0a19560cd8e1e10aebadd01f/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/644f8ed6acb8e1de03f3af14a2f5bc314666f5a819fab2ad277b8ed76798970d/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/9fc703fbe04dd6a8a230bd0bfb4a3b98e94d81676ed3844f0ab4316bdb50cb84/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/066a56846b3752fadf53bfb1d60d2edd39e52bc234fca909c5f320e5fede9528/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/67bdf6a6e1f676c192bfcc8205def50d936e970b7a940ec7feec125201418579/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/79adc9155aee9c364f0ba284ba045af269987d0fad82768399ca08b1c4653acf/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/d9fc7e0ee626c0d578c849531d3d90595465f46799627c2c2243e4b01408af0d/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/08b80fe7e93c50fbba7e444e26359ca35623fd6d6fe7fd3e069ee4417f8b50a0/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/89028c3295204009690a16652d06efeb6722394f6bba0550268d8cbf58a0b933/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/3e75d202c81a84a1ce956d367a269345577b24a7142c6724b0132ad5c2f96cdc/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/fb0f58c9d0b849909d3502dab1437fb07d8798728197aa0402d69dfbba05bb7f/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/e661091ac6d34c562d9bb11acc8512570ac9187983f82767e244c60d586f2828/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/d2764a13bbe826f8c85c1cb726abecf8ca5164611cb8f673424706b376659a37/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/4d26fb7a187f3c8bab1c771715876f25778a68ec5a13b87d4b34a424d45b120b/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/f4f50ca580b4344df02daf801c167cc0ebcc558e154499cc11044a3023b51df7/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/ded3ca117106498899e9c2ca1538c6ad521a13e1d2fb69343ee6196a687b3779/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/632becaffd8667e12383f0a1dbc4f2871b10b1e9a904f37f87de0ad8ae7eaad7/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-206:{mountpoint:/var/lib/containers/storage/overlay/c0b30675141205e277a27f4179cf6ffa82ab3ad0e3186b2931fe1cfba8826587/merged major:0 minor:206 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/c1f723cca84c691bf3f2007a6cd257dffc1d26eaa80770067bc4576800520f06/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/0a89aaa9b76dad8e37e43a79556dbc0c53f72ae27c248893377d295b613e7197/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/2dbb15d264dc546b7308001f528ab70410a3ecbb4c02237b5db84769266a355b/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/363976e28c37cad9a7b997661b9bc9ae8b1647e44c75d1cdb61d12c6a25d806d/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/1e498a86f4022dfb19d771df2df23602531f74f29ffe306dfc394551e0cc1066/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/cd471be49e0f537d9f51fc7cb2fc3bc814620a764c309e6ee0143113d5748a85/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/0faa0b6fdfe955f0eaa86c9793e76c2a14dd3a9059493cf431df71051ce95d9a/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/52dbdc50a6bd8d87d549b6850b867e5164294aab6f0646338201b7f77f924051/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/1dd7b57b0ded89afe11f6ab879e4cd7b217b9a17e6e4252744b859a7084228c5/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/91b92ee90e2b6d77cfbd6e38909b506002327bd4490b1e1a55eda83e7ccb7983/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/68bbd9310e20be2178eb29f82c3d1041627c28f06fa4192f81a2ed462ee4628b/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/d0aa59149a48cd25e60a9c9570c2cd917011dbbf0ca40977ba627d9491f6b2ea/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/0d3cc4635a9172ac0c50e419f0fdc0b2296b48e0aa0eefa03ce5dcfce5ce2c1e/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/289a3b6dbfa9fd29ee51777a707989cc73ad2c35c137a14580581264094be60d/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/a328d519da6be944a516954137a669e7700446cb4024d4788cbef998a16c0591/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/91d09021d52c70cf074bb8e9767c18fc8569ede2cb0ce3fa2339fee85f1274f5/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/0bea453d4b23f47727512bee6bc732bcc5febb0dc94cb452d4d225f847f29bc7/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5b426bbeb6881cb993c8b9cf480cea01a115622f5eb83e9763951581d3b88872/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/33326771897b6fea1280f577a5c9fc9771946f7b3bd382efdf4849ebf60b1422/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/f0b19491dbbd1002740e6c27dd6826df26713dace8c10a31ac65161956f77144/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/2aa42f57fa84ee6c32d6978ae93be2eb752289b8533254659b607bed82d9a1e4/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.287929    4549 manager.go:217] Machine: {Timestamp:2026-10-30 21:00:17.28744211 +0000 UTC m=+0.061842039 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105074 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bfebc498d6147af716fec012ede1f SystemUUID:ec2bfebc-498d-6147-af71-6fec012ede1f BootID:02fa2206-b817-4f89-81aa-63ed791d13a3 Filesystems:[{Device:/var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6a138138177663bef929e7995f737dcf58cd9fe81d879c09be8f19fa8881aa2d/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8dd7870b8562fc76857466170170dfd084805c914578a61d0ca063c0fddf9f68/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volumes/kubernetes.io~projected/kube-api-access-56x7c DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c0c2a2ca93400ffe63d4907e23681668496d0a1f883e9a14cb0ee26ef26c25cc/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3b759b7b-3f86-427d-a1f7-6382fbe96464/volumes/kubernetes.io~projected/kube-api-access-w4z66 DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f03c0a200eb5a467d2bb48b933fae1f872cb252ef7f134daef33470e4d4aee6/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c21d3d9d73a6e5b6c55092a1bc5ad4a92133f56cc7bbf9b2595ecee6936c32a/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3c151da3-d2a5-46fe-97bc-51d030c13470/volumes/kubernetes.io~projected/kube-api-access-rcrlc DeviceMajor:0 DeviceMinor:204 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-206 DeviceMajor:0 DeviceMinor:206 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da/volumes/kubernetes.io~projected/kube-api-access-pq4x6 DeviceMajor:0 DeviceMinor:234 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/458e6a65-37ba-42b3-a21a-a411d302093c/volumes/kubernetes.io~projected/kube-api-access-ftdkm DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82658ff34bd443f5e787f55534daea67bda3b2c1967f3f09a6c11d110bd355f7/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c645ee22-4c7d-443e-ae3a-0d50999f7c05/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2171b3a9e40442cf7c8e7639ef9561fc522261540f6093afb320f25cd04f0b15/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e5a2f410-97fd-4998-962c-dd2f8cd3281d/volumes/kubernetes.io~projected/kube-api-access-vrcms DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4fc96670-a561-4f89-896a-a1b8369560a8/volumes/kubernetes.io~projected/kube-api-access-799cl DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/faa297153eef1bb2715561e5ac4f90f4f18e1749aa8c1c20233ceaba1d1eb2c6/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5649b8ede134632a02974cff8ed6bb58e660b0a0d34fbc7c27413acfbb955023/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9b1e370a131a675c084cda94700df0380d2761a34abe46fa1eeeece7f81b0d58/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/aac5c587-aab5-48db-9fc7-adc2f648ef45/volumes/kubernetes.io~projected/kube-api-access-tx6sr DeviceMajor:0 DeviceMinor:218 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6ef41467-164c-4bc7-8870-d44ec9c655a9/volumes/kubernetes.io~projected/kube-api-access-hnvzb DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a4e28884-7c4b-4a04-a21c-ba0214022cbe/volumes/kubernetes.io~projected/kube-api-access-4x9zv DeviceMajor:0 DeviceMinor:210 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b346fb18-266d-44b4-998e-afdaf5ae0085/volumes/kubernetes.io~projected/kube-api-access-w5n8v DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22/volumes/kubernetes.io~projected/kube-api-access-bvdsw DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a7034506-9c4f-4252-8564-31d0bbe36a16/volumes/kubernetes.io~projected/kube-api-access-mqlkc DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0257b20fd2c3c06dc7861f3b48f1e244bd16fec6132dfeeb85f58f1d3ed93fe8/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab409ac1b3288011539755e8125da03393c2f50e53b065fa3bd76f54d17c7f62/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c645ee22-4c7d-443e-ae3a-0d50999f7c05/volumes/kubernetes.io~projected/kube-api-access-xcpkx DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4c1d2b37-38ac-4aca-90d3-144d6537006f/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:225 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b0cf3a14-78d5-4493-a209-25a4f0233d59/volumes/kubernetes.io~projected/kube-api-access-pxj6c DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/58190e0cfa3d22d6da131d92e0a19daf2148448ca0884ba7902d061cfaa53a8b/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4bc6f18b-e6e7-4909-8321-f7d14cf54ead/volumes/kubernetes.io~projected/kube-api-access-bbjgn DeviceMajor:0 DeviceMinor:205 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/aac5c587-aab5-48db-9fc7-adc2f648ef45/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:217 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/81eb3986-da71-4e0e-862b-c84d3f33815c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:2171b3a9e40442c MacAddress:3a:31:a8:52:6f:a4 Speed:10000 Mtu:8901} {Name:5d3e65bd3166b3f MacAddress:26:a9:55:d9:ca:48 Speed:10000 Mtu:8901} {Name:9b1e370a131a675 MacAddress:3e:ea:54:6b:78:1b Speed:10000 Mtu:8901} {Name:ab409ac1b328801 MacAddress:3e:23:e5:8a:e8:bc Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:fb:3e:59:e1 Speed:0 Mtu:9001} {Name:br-int MacAddress:22:fd:1b:70:1f:a5 Speed:0 Mtu:8901} {Name:c0c2a2ca93400ff MacAddress:a6:b3:fe:d1:75:8f Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:fb:3e:59:e1 Speed:0 Mtu:9001} {Name:faa297153eef1bb MacAddress:a2:7d:f2:0f:ee:5d Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:b6:73:fe:30:7d:5f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:7e:b3:8c:4b:3c:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.288398    4549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.288471    4549 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.288809    4549 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.288854    4549 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-179.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289174    4549 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289193    4549 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289228    4549 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289256    4549 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289464    4549 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289764    4549 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289929    4549 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289963    4549 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289977    4549 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.289992    4549 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.290001    4549 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:00:17.291016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.290010    4549 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:00:17.292406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.292381    4549 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:00:17.292406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.292406    4549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:00:17.293041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.293022    4549 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:00:17.295084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295070    4549 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:00:17.295084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295084    4549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:00:17.295197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295188    4549 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:00:17.295390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295378    4549 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:00:17.295421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295408    4549 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:00:17.295512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295504    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:00:17.295555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295518    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:00:17.295555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295524    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:00:17.295555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295530    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:00:17.295555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295536    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:00:17.295555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295542    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:00:17.295555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295557    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295564    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295571    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295577    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295591    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295600    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295635    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:00:17.295774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.295642    4549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:00:17.299254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.299238    4549 server.go:1263] "Started kubelet"
Oct 30 21:00:17.299690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.299314    4549 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:00:17.299690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.299373    4549 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:00:17.299690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.299486    4549 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:00:17.299961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.299812    4549 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:00:17.299891 ip-10-0-140-179 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:00:17.301196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.301180    4549 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.302753    4549 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.302906    4549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.302920    4549 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.302911    4549 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.302960    4549 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.303037    4549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:54:40 +0000 UTC" deadline="2028-05-10 08:36:53.233410043 +0000 UTC"
Oct 30 21:00:17.303126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.303074    4549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13379h36m35.930339331s"
Oct 30 21:00:17.304954 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:17.303428    4549 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-179.ec2.internal\" not found"
Oct 30 21:00:17.304954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304529    4549 factory.go:55] Registering systemd factory
Oct 30 21:00:17.304954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304545    4549 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:00:17.304954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304863    4549 factory.go:153] Registering CRI-O factory
Oct 30 21:00:17.304954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304879    4549 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304971    4549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304995    4549 factory.go:103] Registering Raw factory
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.304983    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="458e6a65-37ba-42b3-a21a-a411d302093c" volumeName="kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305017    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="458e6a65-37ba-42b3-a21a-a411d302093c" volumeName="kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305032    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" volumeName="kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305045    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4e28884-7c4b-4a04-a21c-ba0214022cbe" volumeName="kubernetes.io/projected/a4e28884-7c4b-4a04-a21c-ba0214022cbe-kube-api-access-4x9zv" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305059    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aac5c587-aab5-48db-9fc7-adc2f648ef45" volumeName="kubernetes.io/secret/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-key" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305070    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0cf3a14-78d5-4493-a209-25a4f0233d59" volumeName="kubernetes.io/projected/b0cf3a14-78d5-4493-a209-25a4f0233d59-kube-api-access-pxj6c" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305099    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c645ee22-4c7d-443e-ae3a-0d50999f7c05" volumeName="kubernetes.io/projected/c645ee22-4c7d-443e-ae3a-0d50999f7c05-kube-api-access-xcpkx" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305112    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c151da3-d2a5-46fe-97bc-51d030c13470" volumeName="kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305126    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="458e6a65-37ba-42b3-a21a-a411d302093c" volumeName="kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305140    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" volumeName="kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305159    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81eb3986-da71-4e0e-862b-c84d3f33815c" volumeName="kubernetes.io/secret/81eb3986-da71-4e0e-862b-c84d3f33815c-agent-certs" seLinuxMountContext=""
Oct 30 21:00:17.305176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305173    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a7034506-9c4f-4252-8564-31d0bbe36a16" volumeName="kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305187    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0cf3a14-78d5-4493-a209-25a4f0233d59" volumeName="kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-tmp" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305202    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0cf3a14-78d5-4493-a209-25a4f0233d59" volumeName="kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-tuned" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305217    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" volumeName="kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-env-overrides" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305238    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b759b7b-3f86-427d-a1f7-6382fbe96464" volumeName="kubernetes.io/projected/3b759b7b-3f86-427d-a1f7-6382fbe96464-kube-api-access-w4z66" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305251    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c1d2b37-38ac-4aca-90d3-144d6537006f" volumeName="kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305263    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81eb3986-da71-4e0e-862b-c84d3f33815c" volumeName="kubernetes.io/configmap/81eb3986-da71-4e0e-862b-c84d3f33815c-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305276    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" volumeName="kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305289    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c151da3-d2a5-46fe-97bc-51d030c13470" volumeName="kubernetes.io/projected/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-api-access-rcrlc" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305301    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22" volumeName="kubernetes.io/projected/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kube-api-access-bvdsw" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305313    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a7034506-9c4f-4252-8564-31d0bbe36a16" volumeName="kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305324    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" volumeName="kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-config" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305336    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" volumeName="kubernetes.io/secret/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305348    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5a2f410-97fd-4998-962c-dd2f8cd3281d" volumeName="kubernetes.io/empty-dir/e5a2f410-97fd-4998-962c-dd2f8cd3281d-tmp-dir" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305359    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5a2f410-97fd-4998-962c-dd2f8cd3281d" volumeName="kubernetes.io/projected/e5a2f410-97fd-4998-962c-dd2f8cd3281d-kube-api-access-vrcms" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305373    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ef41467-164c-4bc7-8870-d44ec9c655a9" volumeName="kubernetes.io/projected/6ef41467-164c-4bc7-8870-d44ec9c655a9-kube-api-access-hnvzb" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305384    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b346fb18-266d-44b4-998e-afdaf5ae0085" volumeName="kubernetes.io/configmap/b346fb18-266d-44b4-998e-afdaf5ae0085-serviceca" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305401    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b346fb18-266d-44b4-998e-afdaf5ae0085" volumeName="kubernetes.io/projected/b346fb18-266d-44b4-998e-afdaf5ae0085-kube-api-access-w5n8v" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305414    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" volumeName="kubernetes.io/projected/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-kube-api-access-56x7c" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305426    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a7034506-9c4f-4252-8564-31d0bbe36a16" volumeName="kubernetes.io/projected/a7034506-9c4f-4252-8564-31d0bbe36a16-kube-api-access-mqlkc" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305440    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c151da3-d2a5-46fe-97bc-51d030c13470" volumeName="kubernetes.io/empty-dir/3c151da3-d2a5-46fe-97bc-51d030c13470-data-volume" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305452    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" volumeName="kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305463    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" volumeName="kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305476    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c645ee22-4c7d-443e-ae3a-0d50999f7c05" volumeName="kubernetes.io/secret/c645ee22-4c7d-443e-ae3a-0d50999f7c05-cert" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305487    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5a2f410-97fd-4998-962c-dd2f8cd3281d" volumeName="kubernetes.io/configmap/e5a2f410-97fd-4998-962c-dd2f8cd3281d-config-volume" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305502    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b759b7b-3f86-427d-a1f7-6382fbe96464" volumeName="kubernetes.io/empty-dir/3b759b7b-3f86-427d-a1f7-6382fbe96464-tmp-dir" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305514    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="458e6a65-37ba-42b3-a21a-a411d302093c" volumeName="kubernetes.io/projected/458e6a65-37ba-42b3-a21a-a411d302093c-kube-api-access-ftdkm" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305526    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fc96670-a561-4f89-896a-a1b8369560a8" volumeName="kubernetes.io/configmap/4fc96670-a561-4f89-896a-a1b8369560a8-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305540    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fc96670-a561-4f89-896a-a1b8369560a8" volumeName="kubernetes.io/projected/4fc96670-a561-4f89-896a-a1b8369560a8-kube-api-access-799cl" seLinuxMountContext=""
Oct 30 21:00:17.305544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305551    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aac5c587-aab5-48db-9fc7-adc2f648ef45" volumeName="kubernetes.io/configmap/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-cabundle" seLinuxMountContext=""
Oct 30 21:00:17.306581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305563    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aac5c587-aab5-48db-9fc7-adc2f648ef45" volumeName="kubernetes.io/projected/aac5c587-aab5-48db-9fc7-adc2f648ef45-kube-api-access-tx6sr" seLinuxMountContext=""
Oct 30 21:00:17.306581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.306077    4549 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09b454b3-3ffa-4639-9fb9-5fbc06ebd0da" volumeName="kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6" seLinuxMountContext=""
Oct 30 21:00:17.306581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.306104    4549 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:00:17.306581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.306113    4549 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:00:17.306581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.305023    4549 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:00:17.306808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.306592    4549 manager.go:319] Starting recovery of all containers
Oct 30 21:00:17.333717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.333687    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:00:17.333717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.333702    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:00:17.334117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.334081    4549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:00:17.335676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.335654    4549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:00:17.335676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.335680    4549 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:00:17.335809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.335700    4549 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:00:17.335809 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:17.335742    4549 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:00:17.345867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.345818    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:00:17.346206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.346175    4549 generic.go:356] "Generic (PLEG): container finished" podID="dabeb57c-4fde-4a6d-a4c8-e1943641d0aa" containerID="dbff679efa30f7d832f9c1178227c8a770b06968488154e1f5542fa95010613a" exitCode=1
Oct 30 21:00:17.351533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.351210    4549 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:00:17.351533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.351383    4549 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:00:17.352622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.352593    4549 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:00:17.362365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.362326    4549 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="72471c4421638e2ed89328a620b293b487558312cd99d0ccc0c33f77427bda2d" exitCode=0
Oct 30 21:00:17.362365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.362360    4549 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="22bcdbef6544e99519ec0db5864652411f8564e52c854a8b1f2901bc35659934" exitCode=0
Oct 30 21:00:17.362365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.362367    4549 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="d6900617e0e0dc622f3ba39e3ea4331cd10f6f64dc1ed0a5afc20b16cccd6a9f" exitCode=0
Oct 30 21:00:17.362365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.362374    4549 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="e7d61e03eca0eb16b88f70a8a2dee17ab9375460c2e9b4bd11de86569158caa5" exitCode=0
Oct 30 21:00:17.362689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.362382    4549 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="d5e8d75c309b375a5724f95ddb8fcad42f012bc1d72f6c87a5d7d0dcded62f7b" exitCode=0
Oct 30 21:00:17.362689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.362388    4549 generic.go:356] "Generic (PLEG): container finished" podID="458e6a65-37ba-42b3-a21a-a411d302093c" containerID="f6e03aa01360d117a80d1f8e39d6d064cac39808ec98e128e71bc9a128db0ddf" exitCode=0
Oct 30 21:00:17.364641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.364616    4549 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:00:17.364765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.364726    4549 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:00:17.366153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.366129    4549 generic.go:356] "Generic (PLEG): container finished" podID="71628ad4075ef9e5c4cd365578233f56" containerID="7a7139692a4795fa3b6488f3d742c1d53df696ecfc1e0b30c5c2a61ed702b640" exitCode=0
Oct 30 21:00:17.385929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.385905    4549 manager.go:324] Recovery completed
Oct 30 21:00:17.402052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402022    4549 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:00:17.402052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402042    4549 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:00:17.402217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402066    4549 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:00:17.402217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402199    4549 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:00:17.402288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402210    4549 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:00:17.402288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402227    4549 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:00:17.402288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402232    4549 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:00:17.402288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402238    4549 policy_none.go:50] "Start"
Oct 30 21:00:17.402288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402246    4549 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:00:17.402288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402256    4549 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:00:17.402452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402382    4549 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:00:17.402452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402389    4549 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:00:17.402452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.402395    4549 policy_none.go:44] "Start"
Oct 30 21:00:17.409016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.408980    4549 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:00:17.412990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.412969    4549 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:00:17.413622 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:17.413604    4549 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:00:17.413665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.413624    4549 server.go:87] "Starting device plugin registration server"
Oct 30 21:00:17.413914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.413902    4549 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:00:17.413946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.413916    4549 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:00:17.414043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.414016    4549 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:00:17.414166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.414127    4549 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:00:17.414166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.414166    4549 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:00:17.414260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.414170    4549 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:00:17.414301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.414212    4549 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:00:17.41415574Z","UUID":"87752c7e-ccd9-43aa-8463-1270bc658982","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:00:17.415582 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:17.415561    4549 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:00:17.415678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.415645    4549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:00:17.415749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.415682    4549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:00:17.435914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.435806    4549 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"]
Oct 30 21:00:17.436441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436400    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal" event={"ID":"0d94f708ebb6ce685e97c708d7e6cc9d","Type":"ContainerStarted","Data":"8910d803d0952c25886cc5aa10f745e07273b99824ea1a2572bf36e88e42be64"}
Oct 30 21:00:17.436529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436454    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal" event={"ID":"0d94f708ebb6ce685e97c708d7e6cc9d","Type":"ContainerStarted","Data":"4435282855cc934cb9a7d010cb508c81cad86d019feab9ea1e57aacdbed25193"}
Oct 30 21:00:17.436529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436475    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" event={"ID":"71628ad4075ef9e5c4cd365578233f56","Type":"ContainerStarted","Data":"3998a26c86bd35061af5d9941e158ea037c43a8b63f2541a0f474e696e666626"}
Oct 30 21:00:17.436529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436496    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" event={"ID":"71628ad4075ef9e5c4cd365578233f56","Type":"ContainerDied","Data":"7a7139692a4795fa3b6488f3d742c1d53df696ecfc1e0b30c5c2a61ed702b640"}
Oct 30 21:00:17.436529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436516    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal" event={"ID":"71628ad4075ef9e5c4cd365578233f56","Type":"ContainerStarted","Data":"1d11324da0b8ed66254aa85e48d03de300b6d5502a9de3dae7a043163a74a6ef"}
Oct 30 21:00:17.436529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436522    4549 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.436679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.436477    4549 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.456426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.456398    4549 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:00:17.456530 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:17.456451    4549 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.456857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.456821    4549 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:00:17.456909 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:17.456883    4549 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.514635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.514597    4549 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:00:17.516082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.516062    4549 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 21:00:17.516193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.516102    4549 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 21:00:17.516193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.516117    4549 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-179.ec2.internal" event="NodeHasSufficientPID"
Oct 30 21:00:17.516193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.516154    4549 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.551370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.551334    4549 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.551594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.551429    4549 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.551711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.551693    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.551766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.551720    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.551766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.551740    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d94f708ebb6ce685e97c708d7e6cc9d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" (UID: \"0d94f708ebb6ce685e97c708d7e6cc9d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.652623 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.652578    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d94f708ebb6ce685e97c708d7e6cc9d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" (UID: \"0d94f708ebb6ce685e97c708d7e6cc9d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.652795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.652634    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.652795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.652652    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.652795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.652581    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d94f708ebb6ce685e97c708d7e6cc9d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" (UID: \"0d94f708ebb6ce685e97c708d7e6cc9d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.652795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.652690    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:17.652795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:17.652748    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71628ad4075ef9e5c4cd365578233f56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" (UID: \"71628ad4075ef9e5c4cd365578233f56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:18.290470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.290435    4549 apiserver.go:52] "Watching apiserver"
Oct 30 21:00:18.328632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.328603    4549 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:00:18.329091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.329069    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-wrhzf","kube-system/konnectivity-agent-8t9kn","openshift-multus/multus-22z8l","openshift-multus/multus-additional-cni-plugins-mmfcw","openshift-ingress-canary/ingress-canary-gvgtv","kube-system/global-pull-secret-syncer-cpdvw","openshift-network-operator/iptables-alerter-rcqw6","openshift-ovn-kubernetes/ovnkube-node-d44wt","openshift-network-diagnostics/network-check-target-hkm5p","openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q","openshift-cluster-node-tuning-operator/tuned-qlqw5","kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal","openshift-image-registry/node-ca-lb6cv","openshift-insights/periodic-gathering-8db7s-5kjq9","openshift-dns/dns-default-h4cm4","openshift-multus/network-metrics-daemon-8pcqj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal","openshift-insights/insights-runtime-extractor-5blgd","openshift-dns/node-resolver-5wsmb"]
Oct 30 21:00:18.332127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.332101    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:18.333692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.333670    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.334055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.334038    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.348330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348302    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:00:18.348544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348525    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348442    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348468    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348563    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348580    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348600    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tzc2p\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348532    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348614    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348315    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-56hzc\""
Oct 30 21:00:18.348627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348629    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348586    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tnqjk\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348532    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348502    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f7r6m\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348499    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348662    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348759    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-hvtl2\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348818    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348862    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4x2zw\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348892    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.348943    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j8v8x\""
Oct 30 21:00:18.349093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349054    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:00:18.349864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349129    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:00:18.349864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349181    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.349864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349236    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b8vzq\""
Oct 30 21:00:18.349864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349268    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.349864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349306    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:00:18.349864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.349366    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:00:18.354063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.354045    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jljmv\""
Oct 30 21:00:18.354926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.354908    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:00:18.355059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.354964    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:00:18.355059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.354999    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.355059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355017    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:00:18.355059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355030    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:00:18.355059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355034    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:00:18.355384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355068    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.355384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355124    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l57v4\""
Oct 30 21:00:18.355384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355216    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.355384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355217    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-79n9v\""
Oct 30 21:00:18.355384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355376    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fz2bf\""
Oct 30 21:00:18.355627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355482    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.355627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355484    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.355627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355502    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.355627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355625    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.355855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.355679    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5jdx7\""
Oct 30 21:00:18.360704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360683    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360758    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bkjxf\""
Oct 30 21:00:18.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360767    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360762    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 21:00:18.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360769    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360817    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360921    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360945    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360971    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.360985    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361009    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gszwc\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361015    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361026    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361027    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361039    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361047    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361057    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m9rs2\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361075    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-xstbj\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361094    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:00:18.361132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361102    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mttl5\""
Oct 30 21:00:18.362016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.361995    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:00:18.377261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.377239    4549 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:18.377384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.377357    4549 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:18.399099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.399066    4549 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:00:18.399240 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:18.399129    4549 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-179.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:18.399723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.399708    4549 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:00:18.399777 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:18.399743    4549 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-179.ec2.internal"
Oct 30 21:00:18.404993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.404974    4549 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:00:18.457760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457722    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-cnibin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457769    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a2f410-97fd-4998-962c-dd2f8cd3281d-tmp-dir\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457795    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdkm\" (UniqueName: \"kubernetes.io/projected/458e6a65-37ba-42b3-a21a-a411d302093c-kube-api-access-ftdkm\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457818    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457866    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-etc-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457889    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysconfig\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457931    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.457995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.457941    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a2f410-97fd-4998-962c-dd2f8cd3281d-tmp-dir\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458069    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-cnibin\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458100    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcpkx\" (UniqueName: \"kubernetes.io/projected/c645ee22-4c7d-443e-ae3a-0d50999f7c05-kube-api-access-xcpkx\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458119    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-registration-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458171    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81eb3986-da71-4e0e-862b-c84d3f33815c-konnectivity-ca\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458241    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458297    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-tuned\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.458362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458357    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-tmp\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458400    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-tuned\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458436    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0cf3a14-78d5-4493-a209-25a4f0233d59-tmp\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458442    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458506    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458528    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458535    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81eb3986-da71-4e0e-862b-c84d3f33815c-konnectivity-ca\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458554    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.458604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458552    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b759b7b-3f86-427d-a1f7-6382fbe96464-hosts-file\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.458889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.458617    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-slash\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.459056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459041    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-host\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.459086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459065    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5n8v\" (UniqueName: \"kubernetes.io/projected/b346fb18-266d-44b4-998e-afdaf5ae0085-kube-api-access-w5n8v\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.459086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459081    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-system-cni-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.459143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459104    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-device-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.459174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459155    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-modprobe-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.459212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459201    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-sys\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.459245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459227    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-dbus\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.459274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459258    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9zv\" (UniqueName: \"kubernetes.io/projected/a4e28884-7c4b-4a04-a21c-ba0214022cbe-kube-api-access-4x9zv\") pod \"migrator-66df5d496b-6zrdn\" (UID: \"a4e28884-7c4b-4a04-a21c-ba0214022cbe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:18.459323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459287    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6sr\" (UniqueName: \"kubernetes.io/projected/aac5c587-aab5-48db-9fc7-adc2f648ef45-kube-api-access-tx6sr\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:18.459398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459385    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.459571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459553    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-sys-fs\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.459604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459589    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4z66\" (UniqueName: \"kubernetes.io/projected/3b759b7b-3f86-427d-a1f7-6382fbe96464-kube-api-access-w4z66\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.459693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459675    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.459734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459692    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-env-overrides\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.459734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459724    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-cni-binary-copy\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.459808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459758    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-socket-dir-parent\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.459808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459785    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-multus\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.459897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459808    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-kubelet\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.459897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459848    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b759b7b-3f86-427d-a1f7-6382fbe96464-tmp-dir\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.459897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459889    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.460027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459922    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b759b7b-3f86-427d-a1f7-6382fbe96464-tmp-dir\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.460027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.459923    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-cni-binary-copy\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460003    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-env-overrides\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.460181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460063    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-bin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460086    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-conf-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460110    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqlkc\" (UniqueName: \"kubernetes.io/projected/a7034506-9c4f-4252-8564-31d0bbe36a16-kube-api-access-mqlkc\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460134    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-cabundle\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:18.460181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460176    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460196    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460213    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvdsw\" (UniqueName: \"kubernetes.io/projected/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kube-api-access-bvdsw\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460228    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-bin\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460243    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-lib-modules\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460267    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnvzb\" (UniqueName: \"kubernetes.io/projected/6ef41467-164c-4bc7-8870-d44ec9c655a9-kube-api-access-hnvzb\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460268    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460298    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-cabundle\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460323    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-socket-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460368    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-netns\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460409    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-var-lib-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.460456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460452    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-system-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460487    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-daemon-config\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460490    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460521    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b346fb18-266d-44b4-998e-afdaf5ae0085-host\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460547    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-systemd-units\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460569    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-systemd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460593    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-var-lib-kubelet\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460616    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-daemon-config\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460645    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460679    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-netns\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460739    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460770    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460809    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc96670-a561-4f89-896a-a1b8369560a8-host-slash\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460838    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460864    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-systemd\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460878    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-run\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460892    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-etc-kubernetes\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460918    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b346fb18-266d-44b4-998e-afdaf5ae0085-serviceca\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460935    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5a2f410-97fd-4998-962c-dd2f8cd3281d-config-volume\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460951    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c151da3-d2a5-46fe-97bc-51d030c13470-crio-socket\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.460967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460966    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c151da3-d2a5-46fe-97bc-51d030c13470-data-volume\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460979    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c1d2b37-38ac-4aca-90d3-144d6537006f-original-pull-secret\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.460981    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxj6c\" (UniqueName: \"kubernetes.io/projected/b0cf3a14-78d5-4493-a209-25a4f0233d59-kube-api-access-pxj6c\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461012    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-k8s-cni-cncf-io\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461071    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-hostroot\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461090    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc96670-a561-4f89-896a-a1b8369560a8-iptables-alerter-script\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461105    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-node-log\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461120    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461135    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-config\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461154    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovn-node-metrics-cert\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461208    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-conf\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461224    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrcms\" (UniqueName: \"kubernetes.io/projected/e5a2f410-97fd-4998-962c-dd2f8cd3281d-kube-api-access-vrcms\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461232    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b346fb18-266d-44b4-998e-afdaf5ae0085-serviceca\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461240    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-key\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461254    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-kubelet\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461269    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-multus-certs\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461284    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461309    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461327    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-799cl\" (UniqueName: \"kubernetes.io/projected/4fc96670-a561-4f89-896a-a1b8369560a8-kube-api-access-799cl\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461343    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461369    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-etc-selinux\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461385    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461396    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5a2f410-97fd-4998-962c-dd2f8cd3281d-config-volume\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461404    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-script-lib\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461420    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81eb3986-da71-4e0e-862b-c84d3f33815c-agent-certs\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461471    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-log-socket\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461486    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-netd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461501    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-kubernetes\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461526    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461542    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461559    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcrlc\" (UniqueName: \"kubernetes.io/projected/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-api-access-rcrlc\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461586    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c645ee22-4c7d-443e-ae3a-0d50999f7c05-cert\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461600    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-ovn\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461616    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56x7c\" (UniqueName: \"kubernetes.io/projected/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-kube-api-access-56x7c\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461631    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-kubelet-config\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461635    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/aac5c587-aab5-48db-9fc7-adc2f648ef45-signing-key\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:18.461896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461795    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.461969    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovn-node-metrics-cert\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462076    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c151da3-d2a5-46fe-97bc-51d030c13470-data-volume\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462593    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-script-lib\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81eb3986-da71-4e0e-862b-c84d3f33815c-agent-certs\") pod \"konnectivity-agent-8t9kn\" (UID: \"81eb3986-da71-4e0e-862b-c84d3f33815c\") " pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462667    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc96670-a561-4f89-896a-a1b8369560a8-iptables-alerter-script\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462687    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c645ee22-4c7d-443e-ae3a-0d50999f7c05-cert\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462784    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/458e6a65-37ba-42b3-a21a-a411d302093c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.463283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.462794    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-ovnkube-config\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.476107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.476080    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdkm\" (UniqueName: \"kubernetes.io/projected/458e6a65-37ba-42b3-a21a-a411d302093c-kube-api-access-ftdkm\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.476259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.476241    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcpkx\" (UniqueName: \"kubernetes.io/projected/c645ee22-4c7d-443e-ae3a-0d50999f7c05-kube-api-access-xcpkx\") pod \"ingress-canary-gvgtv\" (UID: \"c645ee22-4c7d-443e-ae3a-0d50999f7c05\") " pod="openshift-ingress-canary/ingress-canary-gvgtv"
Oct 30 21:00:18.476920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.476895    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6sr\" (UniqueName: \"kubernetes.io/projected/aac5c587-aab5-48db-9fc7-adc2f648ef45-kube-api-access-tx6sr\") pod \"service-ca-5fffc97bdf-wrhzf\" (UID: \"aac5c587-aab5-48db-9fc7-adc2f648ef45\") " pod="openshift-service-ca/service-ca-5fffc97bdf-wrhzf"
Oct 30 21:00:18.477193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.477173    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4z66\" (UniqueName: \"kubernetes.io/projected/3b759b7b-3f86-427d-a1f7-6382fbe96464-kube-api-access-w4z66\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.477317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.477293    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5n8v\" (UniqueName: \"kubernetes.io/projected/b346fb18-266d-44b4-998e-afdaf5ae0085-kube-api-access-w5n8v\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.477375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.477330    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9zv\" (UniqueName: \"kubernetes.io/projected/a4e28884-7c4b-4a04-a21c-ba0214022cbe-kube-api-access-4x9zv\") pod \"migrator-66df5d496b-6zrdn\" (UID: \"a4e28884-7c4b-4a04-a21c-ba0214022cbe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6zrdn"
Oct 30 21:00:18.477497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.477480    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqlkc\" (UniqueName: \"kubernetes.io/projected/a7034506-9c4f-4252-8564-31d0bbe36a16-kube-api-access-mqlkc\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.477680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.477665    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvdsw\" (UniqueName: \"kubernetes.io/projected/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kube-api-access-bvdsw\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.477715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.477677    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnvzb\" (UniqueName: \"kubernetes.io/projected/6ef41467-164c-4bc7-8870-d44ec9c655a9-kube-api-access-hnvzb\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:18.479363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.479294    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-799cl\" (UniqueName: \"kubernetes.io/projected/4fc96670-a561-4f89-896a-a1b8369560a8-kube-api-access-799cl\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.479529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.479500    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrcms\" (UniqueName: \"kubernetes.io/projected/e5a2f410-97fd-4998-962c-dd2f8cd3281d-kube-api-access-vrcms\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.479608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.479540    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxj6c\" (UniqueName: \"kubernetes.io/projected/b0cf3a14-78d5-4493-a209-25a4f0233d59-kube-api-access-pxj6c\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.479764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.479692    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcrlc\" (UniqueName: \"kubernetes.io/projected/3c151da3-d2a5-46fe-97bc-51d030c13470-kube-api-access-rcrlc\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.479932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.479889    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq4x6\" (UniqueName: \"kubernetes.io/projected/09b454b3-3ffa-4639-9fb9-5fbc06ebd0da-kube-api-access-pq4x6\") pod \"network-check-target-hkm5p\" (UID: \"09b454b3-3ffa-4639-9fb9-5fbc06ebd0da\") " pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:18.480088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.480003    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56x7c\" (UniqueName: \"kubernetes.io/projected/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-kube-api-access-56x7c\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.482635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.482610    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") pod \"periodic-gathering-8db7s-5kjq9\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") " pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:00:18.562466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562371    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-socket-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.562466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562409    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-netns\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562424    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-var-lib-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562441    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-system-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562501    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-system-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562507    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-netns\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562526    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-var-lib-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562517    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-socket-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562519    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b346fb18-266d-44b4-998e-afdaf5ae0085-host\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562557    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b346fb18-266d-44b4-998e-afdaf5ae0085-host\") pod \"node-ca-lb6cv\" (UID: \"b346fb18-266d-44b4-998e-afdaf5ae0085\") " pod="openshift-image-registry/node-ca-lb6cv"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562574    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-systemd-units\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562597    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-systemd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562615    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-systemd-units\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562616    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-var-lib-kubelet\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562642    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-netns\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562643    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-var-lib-kubelet\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562658    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562695    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-netns\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562697    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc96670-a561-4f89-896a-a1b8369560a8-host-slash\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562716    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc96670-a561-4f89-896a-a1b8369560a8-host-slash\") pod \"iptables-alerter-rcqw6\" (UID: \"4fc96670-a561-4f89-896a-a1b8369560a8\") " pod="openshift-network-operator/iptables-alerter-rcqw6"
Oct 30 21:00:18.562776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562691    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-systemd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562800    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:18.562814    4549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562870    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-systemd\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:18.562933    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:19.062907905 +0000 UTC m=+1.837307839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : secret "metrics-daemon-secret" not found
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562818    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-systemd\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562958    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-run\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562963    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562973    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-etc-kubernetes\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562990    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c151da3-d2a5-46fe-97bc-51d030c13470-crio-socket\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.562996    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-run\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563009    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-k8s-cni-cncf-io\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563023    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-hostroot\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563030    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-etc-kubernetes\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563039    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-node-log\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563055    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-k8s-cni-cncf-io\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563060    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563059    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c151da3-d2a5-46fe-97bc-51d030c13470-crio-socket\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563066    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-hostroot\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563076    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-conf\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563089    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-node-log\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563093    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-kubelet\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563114    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-multus-certs\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563136    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-run-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563141    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-kubelet\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563144    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563176    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-run-multus-certs\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563181    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-etc-selinux\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563193    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysctl-conf\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563209    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563222    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-etc-selinux\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:18.563222    4549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563236    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-log-socket\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563251    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-netd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:18.563276    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:19.06325965 +0000 UTC m=+1.837659566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563288    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563300    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-log-socket\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563294    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-kubernetes\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563278    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-netd\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563337    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563345    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-kubernetes\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563366    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-ovn\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563377    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-cni-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563393    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-kubelet-config\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563417    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-cnibin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563440    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-ovn\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563442    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563465    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-etc-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563474    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-cnibin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563472    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-kubelet-config\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563488    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysconfig\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563504    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-etc-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563490    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563514    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-sysconfig\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563513    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-cnibin\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563547    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-registration-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563549    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-cnibin\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563566    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563584    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.563556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563596    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-run-openvswitch\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563602    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-registration-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563604    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b759b7b-3f86-427d-a1f7-6382fbe96464-hosts-file\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563622    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-slash\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563636    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-host\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563651    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-system-cni-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563671    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-device-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563694    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-slash\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563702    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-host\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563695    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-modprobe-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563725    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-system-cni-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563734    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-sys\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563737    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b759b7b-3f86-427d-a1f7-6382fbe96464-hosts-file\") pod \"node-resolver-5wsmb\" (UID: \"3b759b7b-3f86-427d-a1f7-6382fbe96464\") " pod="openshift-dns/node-resolver-5wsmb"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563751    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-device-dir\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563766    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-sys\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563763    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-dbus\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563791    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-etc-modprobe-d\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563793    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c1d2b37-38ac-4aca-90d3-144d6537006f-dbus\") pod \"global-pull-secret-syncer-cpdvw\" (UID: \"4c1d2b37-38ac-4aca-90d3-144d6537006f\") " pod="kube-system/global-pull-secret-syncer-cpdvw"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563803    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-sys-fs\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563855    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-socket-dir-parent\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563878    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-multus\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563882    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22-sys-fs\") pod \"aws-ebs-csi-driver-node-q542q\" (UID: \"6d7fa2f1-b578-4a7c-87d6-3b2d018e6b22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q542q"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563909    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-kubelet\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563915    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-socket-dir-parent\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563935    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-bin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563947    4549 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563957    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-bin\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563989    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-conf-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563958    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-multus-conf-dir\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563937    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-cni-multus\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.564015    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.564047    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-bin\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.564068    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-lib-modules\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.564117    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/458e6a65-37ba-42b3-a21a-a411d302093c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mmfcw\" (UID: \"458e6a65-37ba-42b3-a21a-a411d302093c\") " pod="openshift-multus/multus-additional-cni-plugins-mmfcw"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.564120    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dabeb57c-4fde-4a6d-a4c8-e1943641d0aa-host-cni-bin\") pod \"ovnkube-node-d44wt\" (UID: \"dabeb57c-4fde-4a6d-a4c8-e1943641d0aa\") " pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.563956    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7034506-9c4f-4252-8564-31d0bbe36a16-host-var-lib-kubelet\") pod \"multus-22z8l\" (UID: \"a7034506-9c4f-4252-8564-31d0bbe36a16\") " pod="openshift-multus/multus-22z8l"
Oct 30 21:00:18.565306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.564158    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0cf3a14-78d5-4493-a209-25a4f0233d59-lib-modules\") pod \"tuned-qlqw5\" (UID: \"b0cf3a14-78d5-4493-a209-25a4f0233d59\") " pod="openshift-cluster-node-tuning-operator/tuned-qlqw5"
Oct 30 21:00:18.566271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.566133    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5a2f410-97fd-4998-962c-dd2f8cd3281d-metrics-tls\") pod \"dns-default-h4cm4\" (UID: \"e5a2f410-97fd-4998-962c-dd2f8cd3281d\") " pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.634669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.634633    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:18.784772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.784745    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:18.879177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.879142    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:18.929344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.929316    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h4cm4"]
Oct 30 21:00:18.932747 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:18.932716    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a2f410_97fd_4998_962c_dd2f8cd3281d.slice/crio-9ebb3b8d4d72618af274d4f7b58c5291092f9c7cfea3a99c4fc9181d66171304 WatchSource:0}: Error finding container 9ebb3b8d4d72618af274d4f7b58c5291092f9c7cfea3a99c4fc9181d66171304: Status 404 returned error can't find the container with id 9ebb3b8d4d72618af274d4f7b58c5291092f9c7cfea3a99c4fc9181d66171304
Oct 30 21:00:18.935629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.935607    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:00:18.996386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:18.996351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.067535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.067501    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:19.067737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.067545    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:19.067737 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:19.067641    4549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:19.067737 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:19.067661    4549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:00:19.067737 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:19.067693    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:20.067680952 +0000 UTC m=+2.842080882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:19.067737 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:19.067720    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:20.067704689 +0000 UTC m=+2.842104620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : secret "metrics-daemon-secret" not found
Oct 30 21:00:19.282501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.282419    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.380676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.380637    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h4cm4" event={"ID":"e5a2f410-97fd-4998-962c-dd2f8cd3281d","Type":"ContainerStarted","Data":"9ebb3b8d4d72618af274d4f7b58c5291092f9c7cfea3a99c4fc9181d66171304"}
Oct 30 21:00:19.419847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.419788    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.546617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.546536    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.681563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.681523    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.872017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:19.871981    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:20.074577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.074544    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:20.074814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.074593    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:20.074814 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:20.074681    4549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:00:20.074814 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:20.074722    4549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:20.074814 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:20.074759    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs podName:6ef41467-164c-4bc7-8870-d44ec9c655a9 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:22.074743439 +0000 UTC m=+4.849143369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs") pod "network-metrics-daemon-8pcqj" (UID: "6ef41467-164c-4bc7-8870-d44ec9c655a9") : secret "metrics-daemon-secret" not found
Oct 30 21:00:20.074814 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:20.074775    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:22.074769133 +0000 UTC m=+4.849169050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:20.083611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.083567    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:20.468126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.468083    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:20.546273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.546247    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:20.546427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.546416    4549 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:00:20.601285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.601247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:20.685658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:20.685624    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:21.231726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.231694    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:21.362617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.362586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:21.388879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.388824    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h4cm4" event={"ID":"e5a2f410-97fd-4998-962c-dd2f8cd3281d","Type":"ContainerStarted","Data":"6b35cdb8aa843150c55eab7b71c7e8798e2888a57d9bbb411f581f78ceb15672"}
Oct 30 21:00:21.389037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.388901    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h4cm4" event={"ID":"e5a2f410-97fd-4998-962c-dd2f8cd3281d","Type":"ContainerStarted","Data":"dc2b25357682c722a8507ab046d401f2a0c4eb6b1a999558cc03d7a50b90b691"}
Oct 30 21:00:21.389037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.389012    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:21.682699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.682658    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:21.876448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:21.876405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:22.073619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.073590    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:22.090779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.090724    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:22.090919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.090794    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:22.090919 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:22.090911    4549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:22.090991 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:22.090974    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls podName:3c151da3-d2a5-46fe-97bc-51d030c13470 nodeName:}" failed. No retries permitted until 2026-10-30 21:00:26.09095595 +0000 UTC m=+8.865355870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5blgd" (UID: "3c151da3-d2a5-46fe-97bc-51d030c13470") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:00:22.093269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.093242    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef41467-164c-4bc7-8870-d44ec9c655a9-metrics-certs\") pod \"network-metrics-daemon-8pcqj\" (UID: \"6ef41467-164c-4bc7-8870-d44ec9c655a9\") " pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:22.232535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.232498    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8pcqj"
Oct 30 21:00:22.288378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.288346    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:22.406046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.405967    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:22.525787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.525752    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8pcqj"]
Oct 30 21:00:22.529378 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:22.529347    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef41467_164c_4bc7_8870_d44ec9c655a9.slice/crio-6f384d749e7d91da6d559254ea3788804beaad545319565302705fe98530a753 WatchSource:0}: Error finding container 6f384d749e7d91da6d559254ea3788804beaad545319565302705fe98530a753: Status 404 returned error can't find the container with id 6f384d749e7d91da6d559254ea3788804beaad545319565302705fe98530a753
Oct 30 21:00:22.533100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:22.533076    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:23.280495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:23.280452    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:23.397682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:23.397646    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8pcqj" event={"ID":"6ef41467-164c-4bc7-8870-d44ec9c655a9","Type":"ContainerStarted","Data":"6f384d749e7d91da6d559254ea3788804beaad545319565302705fe98530a753"}
Oct 30 21:00:23.679089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:23.679051    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:23.893373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:23.893337    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:24.114419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:24.114383    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:24.406589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:24.406556    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8pcqj" event={"ID":"6ef41467-164c-4bc7-8870-d44ec9c655a9","Type":"ContainerStarted","Data":"ed7e2738db700d8c5b941fbe7675b97fc9a8bb367af0e77c559b176549950b63"}
Oct 30 21:00:24.407028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:24.406641    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8pcqj" event={"ID":"6ef41467-164c-4bc7-8870-d44ec9c655a9","Type":"ContainerStarted","Data":"7638e7235bb282e7c7732461caf5e182e904ced489186bb646e725e4facf1ae5"}
Oct 30 21:00:24.490627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:24.490587    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:24.872641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:24.872599    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:25.087303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:25.087266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:26.115908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:26.115868    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:26.118278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:26.118256    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c151da3-d2a5-46fe-97bc-51d030c13470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5blgd\" (UID: \"3c151da3-d2a5-46fe-97bc-51d030c13470\") " pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:26.135311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:26.135272    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5blgd"
Oct 30 21:00:26.453648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:26.453565    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5blgd"]
Oct 30 21:00:26.457270 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:26.457242    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c151da3_d2a5_46fe_97bc_51d030c13470.slice/crio-e2cdc2c36c7723c774144d091d0e5d5e7850d3f23deff2b08e8237ee0d3d41f1 WatchSource:0}: Error finding container e2cdc2c36c7723c774144d091d0e5d5e7850d3f23deff2b08e8237ee0d3d41f1: Status 404 returned error can't find the container with id e2cdc2c36c7723c774144d091d0e5d5e7850d3f23deff2b08e8237ee0d3d41f1
Oct 30 21:00:27.420691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:27.420658    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5blgd" event={"ID":"3c151da3-d2a5-46fe-97bc-51d030c13470","Type":"ContainerStarted","Data":"5c6b241e5fc08f8bbf2c2bb08b8928ceda584f3e8b9e8545f370c9996a371b08"}
Oct 30 21:00:27.421154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:27.420714    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5blgd" event={"ID":"3c151da3-d2a5-46fe-97bc-51d030c13470","Type":"ContainerStarted","Data":"e2cdc2c36c7723c774144d091d0e5d5e7850d3f23deff2b08e8237ee0d3d41f1"}
Oct 30 21:00:28.427460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:28.427424    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5blgd" event={"ID":"3c151da3-d2a5-46fe-97bc-51d030c13470","Type":"ContainerStarted","Data":"cb4bfdbd8f224b458d4f076f438aee6ad88aeba35c92dd56df21aff8b925498f"}
Oct 30 21:00:30.434929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.434890    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5blgd" event={"ID":"3c151da3-d2a5-46fe-97bc-51d030c13470","Type":"ContainerStarted","Data":"b533843d584d075f8f6207b8299e9854e640259ac7224517d64522ae27002d2a"}
Oct 30 21:00:30.569882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.569483    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-75j9w-wvz7c"]
Oct 30 21:00:30.587416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.587383    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.596499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.596470    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-ccg5n"]
Oct 30 21:00:30.611496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.611469    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-79664cd4c9-jmd25"]
Oct 30 21:00:30.611676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.611632    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.627332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.627304    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-zz9bk\""
Oct 30 21:00:30.630479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.630454    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:00:30.630630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.630482    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:00:30.633440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.633403    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-75j9w-wvz7c"]
Oct 30 21:00:30.633440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.633443    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-ccg5n"]
Oct 30 21:00:30.633628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.633464    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79664cd4c9-jmd25"]
Oct 30 21:00:30.633628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.633543    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.656048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.656012    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:00:30.656048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.656036    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:00:30.656048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.656012    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:00:30.709304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.709223    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:30.709895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.709879    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"]
Oct 30 21:00:30.725708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.725679    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7755d75f65-wh86x"]
Oct 30 21:00:30.725894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.725790    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:30.727697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.727664    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:00:30.727697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.727676    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jqkvq\""
Oct 30 21:00:30.727697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.727675    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:00:30.736699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.736678    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"]
Oct 30 21:00:30.736808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.736759    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.740870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.740779    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7755d75f65-wh86x"]
Oct 30 21:00:30.743315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743285    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.743461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743324    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.743461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743349    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.743574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743479    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.743574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743514    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.743574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743550    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmjg6\" (UniqueName: \"kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.743677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743653    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.743715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743693    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d644\" (UniqueName: \"kubernetes.io/projected/1987d310-712c-4d63-b06c-9ac7aa3f961c-kube-api-access-4d644\") pod \"downloads-6dd789c8b7-ccg5n\" (UID: \"1987d310-712c-4d63-b06c-9ac7aa3f961c\") " pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.743759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743726    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.743810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743757    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnmr5\" (UniqueName: \"kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.743959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743900    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1987d310-712c-4d63-b06c-9ac7aa3f961c-tmp\") pod \"downloads-6dd789c8b7-ccg5n\" (UID: \"1987d310-712c-4d63-b06c-9ac7aa3f961c\") " pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.743959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.743939    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.744088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.744004    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.765015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.764985    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 21:00:30.828471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.828423    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4zrzn\""
Oct 30 21:00:30.831745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.828482    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 21:00:30.831745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.828568    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 21:00:30.831745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.828591    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 21:00:30.831983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.828977    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 21:00:30.831983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.829401    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q5gbc\""
Oct 30 21:00:30.844421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844394    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726bbf8e-ffe8-4f24-8600-645b2ba16089-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-d5lhp\" (UID: \"726bbf8e-ffe8-4f24-8600-645b2ba16089\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:30.844559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844429    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37a58eec-af4e-45a4-b6ad-a1542049b08c-installation-pull-secrets\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.844559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844450    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.844559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844472    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.844680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844657    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37a58eec-af4e-45a4-b6ad-a1542049b08c-registry-certificates\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.844731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844702    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.844766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844730    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.844810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844763    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmjg6\" (UniqueName: \"kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.844810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844793    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.844950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844816    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d644\" (UniqueName: \"kubernetes.io/projected/1987d310-712c-4d63-b06c-9ac7aa3f961c-kube-api-access-4d644\") pod \"downloads-6dd789c8b7-ccg5n\" (UID: \"1987d310-712c-4d63-b06c-9ac7aa3f961c\") " pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.844950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844864    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.845047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.844986    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.845144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845116    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.845232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845148    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnmr5\" (UniqueName: \"kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.845232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845186    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37a58eec-af4e-45a4-b6ad-a1542049b08c-ca-trust-extracted\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.845364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845226    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37a58eec-af4e-45a4-b6ad-a1542049b08c-image-registry-private-configuration\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.845364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845272    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37a58eec-af4e-45a4-b6ad-a1542049b08c-trusted-ca\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.845364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845339    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1987d310-712c-4d63-b06c-9ac7aa3f961c-tmp\") pod \"downloads-6dd789c8b7-ccg5n\" (UID: \"1987d310-712c-4d63-b06c-9ac7aa3f961c\") " pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.845364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845372    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.845556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845379    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-registry-tls\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.845556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845413    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.845556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845442    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.845556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845471    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-bound-sa-token\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.845556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845501    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69kx5\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-kube-api-access-69kx5\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.845556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845529    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.845875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845601    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1987d310-712c-4d63-b06c-9ac7aa3f961c-tmp\") pod \"downloads-6dd789c8b7-ccg5n\" (UID: \"1987d310-712c-4d63-b06c-9ac7aa3f961c\") " pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.845875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.845851    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.846160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.846142    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.846373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.846353    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.846410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.846378    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.847731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.847711    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.847820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.847774    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.864518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.864480    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d644\" (UniqueName: \"kubernetes.io/projected/1987d310-712c-4d63-b06c-9ac7aa3f961c-kube-api-access-4d644\") pod \"downloads-6dd789c8b7-ccg5n\" (UID: \"1987d310-712c-4d63-b06c-9ac7aa3f961c\") " pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.864690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.864674    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmjg6\" (UniqueName: \"kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6\") pod \"console-79664cd4c9-jmd25\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") " pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.864728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.864676    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnmr5\" (UniqueName: \"kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5\") pod \"periodic-gathering-75j9w-wvz7c\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") " pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.880625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.880596    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:30.912648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.912606    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:00:30.943672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.943640    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:30.946927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.946899    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37a58eec-af4e-45a4-b6ad-a1542049b08c-ca-trust-extracted\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.946951    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37a58eec-af4e-45a4-b6ad-a1542049b08c-image-registry-private-configuration\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.946970    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37a58eec-af4e-45a4-b6ad-a1542049b08c-trusted-ca\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.946988    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-registry-tls\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.947011    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-bound-sa-token\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.947038    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69kx5\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-kube-api-access-69kx5\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.947073    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726bbf8e-ffe8-4f24-8600-645b2ba16089-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-d5lhp\" (UID: \"726bbf8e-ffe8-4f24-8600-645b2ba16089\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:30.947238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.947098    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37a58eec-af4e-45a4-b6ad-a1542049b08c-installation-pull-secrets\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.947127    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37a58eec-af4e-45a4-b6ad-a1542049b08c-registry-certificates\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.947429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.947241    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37a58eec-af4e-45a4-b6ad-a1542049b08c-ca-trust-extracted\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.948234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.948204    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37a58eec-af4e-45a4-b6ad-a1542049b08c-trusted-ca\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.948388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.948355    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37a58eec-af4e-45a4-b6ad-a1542049b08c-registry-certificates\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.949901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.949879    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-registry-tls\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.950342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.950319    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37a58eec-af4e-45a4-b6ad-a1542049b08c-image-registry-private-configuration\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.950537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.950512    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/726bbf8e-ffe8-4f24-8600-645b2ba16089-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-d5lhp\" (UID: \"726bbf8e-ffe8-4f24-8600-645b2ba16089\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:30.950767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.950750    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37a58eec-af4e-45a4-b6ad-a1542049b08c-installation-pull-secrets\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.966770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.966704    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:30.969436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.969404    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-bound-sa-token\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:30.969542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:30.969520    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69kx5\" (UniqueName: \"kubernetes.io/projected/37a58eec-af4e-45a4-b6ad-a1542049b08c-kube-api-access-69kx5\") pod \"image-registry-7755d75f65-wh86x\" (UID: \"37a58eec-af4e-45a4-b6ad-a1542049b08c\") " pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:31.038348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.037224    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:31.051424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.051396    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:31.070097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.070055    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:31.234209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.234183    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-75j9w-wvz7c"]
Oct 30 21:00:31.236980 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:31.236955    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55057b6_d0b0_4be8_a41c_a6e243e2b990.slice/crio-8db7ee73cb0ccb95df69a139817c9ea598b71cbbd50be57e9da9001aeb6af65f WatchSource:0}: Error finding container 8db7ee73cb0ccb95df69a139817c9ea598b71cbbd50be57e9da9001aeb6af65f: Status 404 returned error can't find the container with id 8db7ee73cb0ccb95df69a139817c9ea598b71cbbd50be57e9da9001aeb6af65f
Oct 30 21:00:31.273905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.273878    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-ccg5n"]
Oct 30 21:00:31.277474 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:31.277449    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1987d310_712c_4d63_b06c_9ac7aa3f961c.slice/crio-e9f6a77fd75212243f70187d2d501e6b5ff83b54dce76747590ab478cca3cfbb WatchSource:0}: Error finding container e9f6a77fd75212243f70187d2d501e6b5ff83b54dce76747590ab478cca3cfbb: Status 404 returned error can't find the container with id e9f6a77fd75212243f70187d2d501e6b5ff83b54dce76747590ab478cca3cfbb
Oct 30 21:00:31.313149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.313120    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79664cd4c9-jmd25"]
Oct 30 21:00:31.317311 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:31.317273    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cebbd8_9bcb_42d6_98f3_6cfbd3b49def.slice/crio-836ddadef6d86368af9bf839e85cfe03044ca7596986c1c4e2d7cc469e3aa6c2 WatchSource:0}: Error finding container 836ddadef6d86368af9bf839e85cfe03044ca7596986c1c4e2d7cc469e3aa6c2: Status 404 returned error can't find the container with id 836ddadef6d86368af9bf839e85cfe03044ca7596986c1c4e2d7cc469e3aa6c2
Oct 30 21:00:31.363815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.363780    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"]
Oct 30 21:00:31.393440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.393396    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7755d75f65-wh86x"]
Oct 30 21:00:31.394454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.394435    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h4cm4"
Oct 30 21:00:31.397209 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:31.397186    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a58eec_af4e_45a4_b6ad_a1542049b08c.slice/crio-b2cc1ddea624879158a53debb978cd630de4312758b49b60dac5785289c84e26 WatchSource:0}: Error finding container b2cc1ddea624879158a53debb978cd630de4312758b49b60dac5785289c84e26: Status 404 returned error can't find the container with id b2cc1ddea624879158a53debb978cd630de4312758b49b60dac5785289c84e26
Oct 30 21:00:31.440736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.440701    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7755d75f65-wh86x" event={"ID":"37a58eec-af4e-45a4-b6ad-a1542049b08c","Type":"ContainerStarted","Data":"b2cc1ddea624879158a53debb978cd630de4312758b49b60dac5785289c84e26"}
Oct 30 21:00:31.442712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.442679    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75j9w-wvz7c" event={"ID":"b55057b6-d0b0-4be8-a41c-a6e243e2b990","Type":"ContainerStarted","Data":"70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c"}
Oct 30 21:00:31.442870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.442732    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75j9w-wvz7c" event={"ID":"b55057b6-d0b0-4be8-a41c-a6e243e2b990","Type":"ContainerStarted","Data":"8db7ee73cb0ccb95df69a139817c9ea598b71cbbd50be57e9da9001aeb6af65f"}
Oct 30 21:00:31.445027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.444955    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-ccg5n" event={"ID":"1987d310-712c-4d63-b06c-9ac7aa3f961c","Type":"ContainerStarted","Data":"e9f6a77fd75212243f70187d2d501e6b5ff83b54dce76747590ab478cca3cfbb"}
Oct 30 21:00:31.446761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.446733    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp" event={"ID":"726bbf8e-ffe8-4f24-8600-645b2ba16089","Type":"ContainerStarted","Data":"f78139947c75c658b8f4b9ec352aa12704281086cc81d5cfabd24f2af13aeca8"}
Oct 30 21:00:31.448955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.448928    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79664cd4c9-jmd25" event={"ID":"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def","Type":"ContainerStarted","Data":"836ddadef6d86368af9bf839e85cfe03044ca7596986c1c4e2d7cc469e3aa6c2"}
Oct 30 21:00:31.480923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:31.480856    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-75j9w-wvz7c" podStartSLOduration=1.48080819 podStartE2EDuration="1.48080819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:31.479720225 +0000 UTC m=+14.254120166" watchObservedRunningTime="2026-10-30 21:00:31.48080819 +0000 UTC m=+14.255208132"
Oct 30 21:00:32.458602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:32.458559    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7755d75f65-wh86x" event={"ID":"37a58eec-af4e-45a4-b6ad-a1542049b08c","Type":"ContainerStarted","Data":"815ee6ee6958915307a372a90c3503ec8108d5610e9f7ba89a3fdd55aef2feff"}
Oct 30 21:00:32.459052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:32.458637    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:32.498954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:32.498892    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7755d75f65-wh86x" podStartSLOduration=2.498872713 podStartE2EDuration="2.498872713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:32.498508416 +0000 UTC m=+15.272908356" watchObservedRunningTime="2026-10-30 21:00:32.498872713 +0000 UTC m=+15.273272653"
Oct 30 21:00:32.774892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:32.774796    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h4cm4_e5a2f410-97fd-4998-962c-dd2f8cd3281d/dns/0.log"
Oct 30 21:00:32.977309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:32.977279    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h4cm4_e5a2f410-97fd-4998-962c-dd2f8cd3281d/kube-rbac-proxy/0.log"
Oct 30 21:00:33.024900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:33.024785    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:33.369611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:33.369573    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wsmb_3b759b7b-3f86-427d-a1f7-6382fbe96464/dns-node-resolver/0.log"
Oct 30 21:00:33.463574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:33.463506    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp" event={"ID":"726bbf8e-ffe8-4f24-8600-645b2ba16089","Type":"ContainerStarted","Data":"e770cf2d36565ea40bb49af35f1131a564d89a01fc0492c39c3cf13f18194b7a"}
Oct 30 21:00:33.500468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:33.500405    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp" podStartSLOduration=2.12136 podStartE2EDuration="3.500391239s" totalImagesPullingTime="1.379031239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:33.50007488 +0000 UTC m=+16.274474819" watchObservedRunningTime="2026-10-30 21:00:33.500391239 +0000 UTC m=+16.274791178"
Oct 30 21:00:34.169877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:34.169844    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7755d75f65-wh86x_37a58eec-af4e-45a4-b6ad-a1542049b08c/registry/0.log"
Oct 30 21:00:34.467511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:34.467428    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:34.473610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:34.473578    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-d5lhp"
Oct 30 21:00:34.569676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:34.569638    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lb6cv_b346fb18-266d-44b4-998e-afdaf5ae0085/node-ca/0.log"
Oct 30 21:00:35.371288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:35.371259    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gvgtv_c645ee22-4c7d-443e-ae3a-0d50999f7c05/serve-healthcheck-canary/0.log"
Oct 30 21:00:35.377454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:35.377420    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:35.472119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:35.472082    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79664cd4c9-jmd25" event={"ID":"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def","Type":"ContainerStarted","Data":"c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684"}
Oct 30 21:00:35.514125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:35.514074    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-79664cd4c9-jmd25" podStartSLOduration=1.878930856 podStartE2EDuration="5.514058165s" totalImagesPullingTime="3.635127309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:35.513911124 +0000 UTC m=+18.288311064" watchObservedRunningTime="2026-10-30 21:00:35.514058165 +0000 UTC m=+18.288458098"
Oct 30 21:00:35.524788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:35.524754    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:35.654254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:35.654166    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:36.412870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:36.412812    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:38.000542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:38.000508    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8t9kn"
Oct 30 21:00:38.196577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:38.196492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:39.791587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:39.791523    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:39.902881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:39.902853    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d44wt"
Oct 30 21:00:39.940326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:39.940293    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.080621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.080591    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2g6cr"]
Oct 30 21:00:40.084818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.084073    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.102159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.102133    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.137322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.137268    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:00:40.137542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.137527    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-595gg\""
Oct 30 21:00:40.137621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.137608    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:00:40.172667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.172631    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:00:40.172667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.172657    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:00:40.172667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.172633    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:00:40.172922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.172860    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:00:40.220231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.220195    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.236363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236334    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdr6j\" (UniqueName: \"kubernetes.io/projected/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-kube-api-access-kdr6j\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236389    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-metrics-client-ca\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236477    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236548    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-textfile\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236581    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236621    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-sys\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236641    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-root\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236681    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-wtmp\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.236789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.236719    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-tls\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337084    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdr6j\" (UniqueName: \"kubernetes.io/projected/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-kube-api-access-kdr6j\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337131    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-metrics-client-ca\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337159    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337188    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-textfile\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337211    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337252    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-sys\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337272    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-root\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337297    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-wtmp\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337329    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-tls\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337504    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-root\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337565    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-sys\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337650    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-textfile\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337687    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-wtmp\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337802    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.337894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.337810    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-metrics-client-ca\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.339874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.339821    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.341137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.341120    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-node-exporter-tls\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.356502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.356468    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdr6j\" (UniqueName: \"kubernetes.io/projected/1a57b7a0-984a-4b56-96af-ba5144c9c7d4-kube-api-access-kdr6j\") pod \"node-exporter-2g6cr\" (UID: \"1a57b7a0-984a-4b56-96af-ba5144c9c7d4\") " pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.398049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.398017    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2g6cr"
Oct 30 21:00:40.412629 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:00:40.412598    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a57b7a0_984a_4b56_96af_ba5144c9c7d4.slice/crio-be090a6b6e06592ceaf00a555629a06e7ffd120b281bb20ee18030a82168ab8c WatchSource:0}: Error finding container be090a6b6e06592ceaf00a555629a06e7ffd120b281bb20ee18030a82168ab8c: Status 404 returned error can't find the container with id be090a6b6e06592ceaf00a555629a06e7ffd120b281bb20ee18030a82168ab8c
Oct 30 21:00:40.413141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.413116    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.493297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.493265    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2g6cr" event={"ID":"1a57b7a0-984a-4b56-96af-ba5144c9c7d4","Type":"ContainerStarted","Data":"be090a6b6e06592ceaf00a555629a06e7ffd120b281bb20ee18030a82168ab8c"}
Oct 30 21:00:40.533759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.533708    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.665400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.665318    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.830748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.830693    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.967333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.966966    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:40.967333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.967051    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:40.968240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.968207    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:40.972685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:40.972667    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:41.164765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:41.164725    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:41.284665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:41.284588    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:41.443202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:41.443164    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:41.499663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:41.499626    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2g6cr" event={"ID":"1a57b7a0-984a-4b56-96af-ba5144c9c7d4","Type":"ContainerStarted","Data":"fde812fd9b26557078a933b4d83f6d469d11c535d0bfb6b5d4348c241e6449dd"}
Oct 30 21:00:41.504748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:41.504721    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:00:42.505641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:42.505600    4549 generic.go:356] "Generic (PLEG): container finished" podID="1a57b7a0-984a-4b56-96af-ba5144c9c7d4" containerID="fde812fd9b26557078a933b4d83f6d469d11c535d0bfb6b5d4348c241e6449dd" exitCode=0
Oct 30 21:00:42.506155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:42.505699    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2g6cr" event={"ID":"1a57b7a0-984a-4b56-96af-ba5144c9c7d4","Type":"ContainerDied","Data":"fde812fd9b26557078a933b4d83f6d469d11c535d0bfb6b5d4348c241e6449dd"}
Oct 30 21:00:42.824261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:42.824230    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:42.953695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:42.953660    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:43.296231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:43.296155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:43.421043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:43.421010    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:43.512802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:43.512771    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2g6cr" event={"ID":"1a57b7a0-984a-4b56-96af-ba5144c9c7d4","Type":"ContainerStarted","Data":"6fa8f5b5254637c5ec47ede136291635596c82c614772a253f0b9a01b05ef5c5"}
Oct 30 21:00:43.513282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:43.512845    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2g6cr" event={"ID":"1a57b7a0-984a-4b56-96af-ba5144c9c7d4","Type":"ContainerStarted","Data":"151f885d9c7235d04cb157f90c7fd28eb85f51b103d6412b2404431852525db4"}
Oct 30 21:00:43.566809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:43.566749    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2g6cr" podStartSLOduration=2.539505596 podStartE2EDuration="3.566736981s" totalImagesPullingTime="941.17417ms" totalInitContainerRuntime="86.057215ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:43.566219812 +0000 UTC m=+26.340619744" watchObservedRunningTime="2026-10-30 21:00:43.566736981 +0000 UTC m=+26.341136920"
Oct 30 21:00:44.726224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:44.726149    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:44.887024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:44.886989    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:45.114167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:45.114137    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:45.317403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:45.317353    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:45.455578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:45.455466    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:45.595933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:45.595891    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:47.039798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:47.039770    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:47.291997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:47.291930    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:47.492167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:47.492127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:48.538219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:48.538133    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-ccg5n" event={"ID":"1987d310-712c-4d63-b06c-9ac7aa3f961c","Type":"ContainerStarted","Data":"3416c780864588dfcb8a429d41ffdb1a5519bfeaeccdcfc9430156039e5290c3"}
Oct 30 21:00:48.538757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:48.538381    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:48.540250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:48.540214    4549 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-ccg5n container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" start-of-body=
Oct 30 21:00:48.540360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:48.540273    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-ccg5n" podUID="1987d310-712c-4d63-b06c-9ac7aa3f961c" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused"
Oct 30 21:00:48.579217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:48.579160    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-ccg5n" podStartSLOduration=1.719411088 podStartE2EDuration="18.579129814s" totalImagesPullingTime="16.859718726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:48.578444087 +0000 UTC m=+31.352844027" watchObservedRunningTime="2026-10-30 21:00:48.579129814 +0000 UTC m=+31.353529754"
Oct 30 21:00:49.044277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:49.044239    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:49.176569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:49.176525    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:49.555301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:49.555270    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-ccg5n"
Oct 30 21:00:50.434172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:50.434134    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:50.552781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:50.552741    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:50.689439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:50.689354    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hkm5p"
Oct 30 21:00:51.126955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:51.126914    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:51.250874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:51.250816    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:52.033107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:52.033035    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:52.239551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:52.239511    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:53.468955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:53.468921    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7755d75f65-wh86x"
Oct 30 21:00:53.639112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:53.639083    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:53.766510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:53.766435    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:53.881936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:53.881894    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:56.353862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:56.353811    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79664cd4c9-jmd25"]
Oct 30 21:00:56.579256 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:00:56.579214    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:00:58.188260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:58.188228    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:58.317689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:00:58.317651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:02.424038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.424007    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-8db7s-5kjq9"]
Oct 30 21:01:02.424578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.424182    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-8db7s-5kjq9" podUID="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" containerName="insights-gathering" containerID="cri-o://5d6b5e91782bc41d9b74e7477ba95447c2c9ca87870109c233fca560b50e7ff6" gracePeriod=30
Oct 30 21:01:02.600733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.600696    4549 generic.go:356] "Generic (PLEG): container finished" podID="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" containerID="5d6b5e91782bc41d9b74e7477ba95447c2c9ca87870109c233fca560b50e7ff6" exitCode=2
Oct 30 21:01:02.600971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.600767    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8db7s-5kjq9" event={"ID":"4bc6f18b-e6e7-4909-8321-f7d14cf54ead","Type":"ContainerDied","Data":"5d6b5e91782bc41d9b74e7477ba95447c2c9ca87870109c233fca560b50e7ff6"}
Oct 30 21:01:02.725269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.725236    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:01:02.816698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.816661    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") pod \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") "
Oct 30 21:01:02.816698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.816704    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") pod \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") "
Oct 30 21:01:02.816939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.816739    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") pod \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") "
Oct 30 21:01:02.816939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.816813    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") pod \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\" (UID: \"4bc6f18b-e6e7-4909-8321-f7d14cf54ead\") "
Oct 30 21:01:02.817137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.817107    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path" pod "4bc6f18b-e6e7-4909-8321-f7d14cf54ead" (UID: "4bc6f18b-e6e7-4909-8321-f7d14cf54ead"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:01:02.817182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.817133    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle" pod "4bc6f18b-e6e7-4909-8321-f7d14cf54ead" (UID: "4bc6f18b-e6e7-4909-8321-f7d14cf54ead"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:02.817182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.817150    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle" pod "4bc6f18b-e6e7-4909-8321-f7d14cf54ead" (UID: "4bc6f18b-e6e7-4909-8321-f7d14cf54ead"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:02.819317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.819291    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn" pod "4bc6f18b-e6e7-4909-8321-f7d14cf54ead" (UID: "4bc6f18b-e6e7-4909-8321-f7d14cf54ead"). InnerVolumeSpecName "kube-api-access-bbjgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:01:02.917448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.917404    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbjgn\" (UniqueName: \"kubernetes.io/projected/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-kube-api-access-bbjgn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:02.917448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.917439    4549 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-archives-path\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:02.917448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.917453    4549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-trusted-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:02.917682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:02.917468    4549 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bc6f18b-e6e7-4909-8321-f7d14cf54ead-service-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:03.234303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.234266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:03.436139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.436100    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:03.604902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.604868    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8db7s-5kjq9" event={"ID":"4bc6f18b-e6e7-4909-8321-f7d14cf54ead","Type":"ContainerDied","Data":"5d3e65bd3166b3f4288dae0f7135ae5ac8d263470e44fe0648b7965dc08d017a"}
Oct 30 21:01:03.605082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.604932    4549 scope.go:122] "RemoveContainer" containerID="5d6b5e91782bc41d9b74e7477ba95447c2c9ca87870109c233fca560b50e7ff6"
Oct 30 21:01:03.605082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.604877    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8db7s-5kjq9"
Oct 30 21:01:03.653625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.653593    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-8db7s-5kjq9"]
Oct 30 21:01:03.666112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:03.666079    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-8db7s-5kjq9"]
Oct 30 21:01:05.343508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:05.341866    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" path="/var/lib/kubelet/pods/4bc6f18b-e6e7-4909-8321-f7d14cf54ead/volumes"
Oct 30 21:01:05.467202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:05.467170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:05.588959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:05.588890    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.414457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:10.414424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.607926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:10.607885    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.881947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:10.881906    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:15.667307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.667270    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-86c86766b4-2gwt9"]
Oct 30 21:01:15.667714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.667507    4549 cpu_manager.go:414] "containerMap: removing container" podUID="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" containerName="insights-gathering"
Oct 30 21:01:15.667714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.667518    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" containerName="insights-gathering"
Oct 30 21:01:15.667714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.667573    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bc6f18b-e6e7-4909-8321-f7d14cf54ead" containerName="insights-gathering"
Oct 30 21:01:15.701308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.701274    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86c86766b4-2gwt9"]
Oct 30 21:01:15.701483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.701384    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.715752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.715720    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:01:15.715752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.715730    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:01:15.715988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.715974    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:01:15.716056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.716037    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:01:15.716105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.716092    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wg5c2\""
Oct 30 21:01:15.768713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.768682    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:01:15.781488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.781462    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:01:15.812042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812012    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9ww6\" (UniqueName: \"kubernetes.io/projected/277cb4db-e7d5-4b31-ba96-1b6de72438dc-kube-api-access-s9ww6\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812058    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-federate-client-tls\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812082    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812103    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-metrics-client-ca\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812169    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-secret-telemeter-client\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812299    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-serving-certs-ca-bundle\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812350    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.812412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.812379    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-telemeter-client-tls\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913329    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913381    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-telemeter-client-tls\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913404    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9ww6\" (UniqueName: \"kubernetes.io/projected/277cb4db-e7d5-4b31-ba96-1b6de72438dc-kube-api-access-s9ww6\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913432    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-federate-client-tls\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913510    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913544    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-metrics-client-ca\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913562    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-secret-telemeter-client\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.913756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.913600    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-serving-certs-ca-bundle\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.914637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.914609    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.914797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.914774    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-serving-certs-ca-bundle\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.914875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.914798    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/277cb4db-e7d5-4b31-ba96-1b6de72438dc-metrics-client-ca\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.916235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.916213    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-federate-client-tls\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.916340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.916316    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.916486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.916470    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-secret-telemeter-client\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.916525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.916518    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/277cb4db-e7d5-4b31-ba96-1b6de72438dc-telemeter-client-tls\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:15.938354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:15.938279    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9ww6\" (UniqueName: \"kubernetes.io/projected/277cb4db-e7d5-4b31-ba96-1b6de72438dc-kube-api-access-s9ww6\") pod \"telemeter-client-86c86766b4-2gwt9\" (UID: \"277cb4db-e7d5-4b31-ba96-1b6de72438dc\") " pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:16.053968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:16.053924    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9"
Oct 30 21:01:16.409995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:16.409971    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86c86766b4-2gwt9"]
Oct 30 21:01:16.412635 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:01:16.412606    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277cb4db_e7d5_4b31_ba96_1b6de72438dc.slice/crio-2680ca2647f74a0ca77486eb295b842fabdff4657cd42db565edc875f91241d0 WatchSource:0}: Error finding container 2680ca2647f74a0ca77486eb295b842fabdff4657cd42db565edc875f91241d0: Status 404 returned error can't find the container with id 2680ca2647f74a0ca77486eb295b842fabdff4657cd42db565edc875f91241d0
Oct 30 21:01:16.649528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:16.649494    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9" event={"ID":"277cb4db-e7d5-4b31-ba96-1b6de72438dc","Type":"ContainerStarted","Data":"2680ca2647f74a0ca77486eb295b842fabdff4657cd42db565edc875f91241d0"}
Oct 30 21:01:19.662081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:19.662044    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9" event={"ID":"277cb4db-e7d5-4b31-ba96-1b6de72438dc","Type":"ContainerStarted","Data":"118bd674547aee6a3dfc6eb87a759ace84c98c2ef6c15545ded10dbf65daedc4"}
Oct 30 21:01:20.670488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.670444    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9" event={"ID":"277cb4db-e7d5-4b31-ba96-1b6de72438dc","Type":"ContainerStarted","Data":"caed373ac614670d9caa42e4e4a0946fc96871eda77dbfa39e792fe8a55f2ebb"}
Oct 30 21:01:20.670921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.670536    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9" event={"ID":"277cb4db-e7d5-4b31-ba96-1b6de72438dc","Type":"ContainerStarted","Data":"e22920f4353d91e8df77df1689c941b713b23065d933a9c41b5c14a8126efc75"}
Oct 30 21:01:20.736513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.736450    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-86c86766b4-2gwt9" podStartSLOduration=1.9960984179999999 podStartE2EDuration="5.736411126s" totalImagesPullingTime="3.740312708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:20.724262456 +0000 UTC m=+63.498662408" watchObservedRunningTime="2026-10-30 21:01:20.736411126 +0000 UTC m=+63.510811065"
Oct 30 21:01:20.901047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.901008    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-67bcf94998-qlwcl"]
Oct 30 21:01:20.920420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.920386    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.926860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.926780    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bcf94998-qlwcl"]
Oct 30 21:01:20.947775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.947742    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:01:20.955030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955001    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955050    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955122    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955189    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955228    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955316    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955379    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:20.955417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:20.955406    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056072    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056109    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056139    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056160    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056183    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056202    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056226    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056264    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056500    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.056890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.056867    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.057085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.057060    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.057164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.057072    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.057440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.057419    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.059000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.058982    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.059060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.059034    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.077072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.077046    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l\") pod \"console-67bcf94998-qlwcl\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") " pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.166287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.166255    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:21.245033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.244947    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:21.295954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.295923    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:21.378393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.378346    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-79664cd4c9-jmd25" podUID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" containerName="console" containerID="cri-o://c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684" gracePeriod=15
Oct 30 21:01:21.423089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.423057    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:21.500796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.500715    4549 patch_prober.go:29] interesting pod/console-79664cd4c9-jmd25 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body=
Oct 30 21:01:21.500796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.500774    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/console-79664cd4c9-jmd25" podUID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused"
Oct 30 21:01:21.567713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.567614    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bcf94998-qlwcl"]
Oct 30 21:01:21.570219 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:01:21.570185    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode341d71e_73c8_4f60_a143_09f2267d0869.slice/crio-ce451723d8df1f41da0fc1c5996f2e6f983646a0c28524be021b7f26f2b18d07 WatchSource:0}: Error finding container ce451723d8df1f41da0fc1c5996f2e6f983646a0c28524be021b7f26f2b18d07: Status 404 returned error can't find the container with id ce451723d8df1f41da0fc1c5996f2e6f983646a0c28524be021b7f26f2b18d07
Oct 30 21:01:21.665076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.665041    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:01:21.675715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.675679    4549 generic.go:356] "Generic (PLEG): container finished" podID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" containerID="c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684" exitCode=0
Oct 30 21:01:21.676140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.675732    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79664cd4c9-jmd25" event={"ID":"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def","Type":"ContainerDied","Data":"c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684"}
Oct 30 21:01:21.676140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.675763    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79664cd4c9-jmd25"
Oct 30 21:01:21.676140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.675884    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79664cd4c9-jmd25" event={"ID":"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def","Type":"ContainerDied","Data":"836ddadef6d86368af9bf839e85cfe03044ca7596986c1c4e2d7cc469e3aa6c2"}
Oct 30 21:01:21.676140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.675915    4549 scope.go:122] "RemoveContainer" containerID="c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684"
Oct 30 21:01:21.677478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.677455    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcf94998-qlwcl" event={"ID":"e341d71e-73c8-4f60-a143-09f2267d0869","Type":"ContainerStarted","Data":"ce451723d8df1f41da0fc1c5996f2e6f983646a0c28524be021b7f26f2b18d07"}
Oct 30 21:01:21.683647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.683630    4549 scope.go:122] "RemoveContainer" containerID="c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684"
Oct 30 21:01:21.684006 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:01:21.683987    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684\": container with ID starting with c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684 not found: ID does not exist" containerID="c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684"
Oct 30 21:01:21.684077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.684019    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684"} err="failed to get container status \"c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684\": rpc error: code = NotFound desc = could not find container \"c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684\": container with ID starting with c269e0531d284857356c9ed4a168187eab5dedd5000d76d68d46f20887074684 not found: ID does not exist"
Oct 30 21:01:21.763567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763489    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6\" (UniqueName: \"kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.763567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763546    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.763812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763574    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.763812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763596    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.763812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763639    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.763812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763699    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.763812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.763742    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config\") pod \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\" (UID: \"d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def\") "
Oct 30 21:01:21.764116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.764092    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:21.764247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.764188    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:21.764477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.764380    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:21.764906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.764883    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:01:21.766077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.766038    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "kube-api-access-zmjg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:01:21.766163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.766125    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:01:21.766592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.766570    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert" pod "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" (UID: "d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:01:21.864523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864488    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmjg6\" (UniqueName: \"kubernetes.io/projected/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-kube-api-access-zmjg6\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:21.864523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864517    4549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-serving-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:21.864523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864530    4549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:21.864777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864539    4549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-oauth-serving-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:21.864777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864547    4549 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-tmp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:21.864777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864556    4549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-config\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:21.864777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:21.864563    4549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def-console-oauth-config\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:22.017703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:22.017614    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79664cd4c9-jmd25"]
Oct 30 21:01:22.021073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:22.021044    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79664cd4c9-jmd25"]
Oct 30 21:01:22.682883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:22.682850    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcf94998-qlwcl" event={"ID":"e341d71e-73c8-4f60-a143-09f2267d0869","Type":"ContainerStarted","Data":"2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b"}
Oct 30 21:01:22.735378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:22.735332    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-67bcf94998-qlwcl" podStartSLOduration=2.735319511 podStartE2EDuration="2.735319511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:22.72312059 +0000 UTC m=+65.497520529" watchObservedRunningTime="2026-10-30 21:01:22.735319511 +0000 UTC m=+65.509719451"
Oct 30 21:01:23.341248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:23.341203    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" path="/var/lib/kubelet/pods/d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def/volumes"
Oct 30 21:01:26.572637 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:01:26.572590    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:01:31.245909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:31.245872    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:31.246377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:31.245955    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:31.250400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:31.250381    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:31.719259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:31.719233    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:01:47.675809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:47.675778    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-75j9w-wvz7c"]
Oct 30 21:01:47.676391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:47.675983    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-75j9w-wvz7c" podUID="b55057b6-d0b0-4be8-a41c-a6e243e2b990" containerName="insights-gathering" containerID="cri-o://70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c" gracePeriod=30
Oct 30 21:01:47.912523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:47.912497    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:01:48.063067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.062991    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5\" (UniqueName: \"kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5\") pod \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") "
Oct 30 21:01:48.063067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.063033    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle\") pod \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") "
Oct 30 21:01:48.063255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.063074    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle\") pod \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") "
Oct 30 21:01:48.063255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.063114    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path\") pod \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\" (UID: \"b55057b6-d0b0-4be8-a41c-a6e243e2b990\") "
Oct 30 21:01:48.063471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.063433    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path" pod "b55057b6-d0b0-4be8-a41c-a6e243e2b990" (UID: "b55057b6-d0b0-4be8-a41c-a6e243e2b990"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:01:48.063583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.063494    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle" pod "b55057b6-d0b0-4be8-a41c-a6e243e2b990" (UID: "b55057b6-d0b0-4be8-a41c-a6e243e2b990"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:48.063583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.063514    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle" pod "b55057b6-d0b0-4be8-a41c-a6e243e2b990" (UID: "b55057b6-d0b0-4be8-a41c-a6e243e2b990"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:48.065330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.065309    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5" pod "b55057b6-d0b0-4be8-a41c-a6e243e2b990" (UID: "b55057b6-d0b0-4be8-a41c-a6e243e2b990"). InnerVolumeSpecName "kube-api-access-gnmr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:01:48.163807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.163768    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnmr5\" (UniqueName: \"kubernetes.io/projected/b55057b6-d0b0-4be8-a41c-a6e243e2b990-kube-api-access-gnmr5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:48.163807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.163798    4549 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-service-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:48.163807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.163807    4549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55057b6-d0b0-4be8-a41c-a6e243e2b990-trusted-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:48.163807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.163816    4549 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b55057b6-d0b0-4be8-a41c-a6e243e2b990-archives-path\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:01:48.774050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.774018    4549 generic.go:356] "Generic (PLEG): container finished" podID="b55057b6-d0b0-4be8-a41c-a6e243e2b990" containerID="70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c" exitCode=2
Oct 30 21:01:48.774503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.774055    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75j9w-wvz7c"
Oct 30 21:01:48.774503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.774080    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75j9w-wvz7c" event={"ID":"b55057b6-d0b0-4be8-a41c-a6e243e2b990","Type":"ContainerDied","Data":"70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c"}
Oct 30 21:01:48.774503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.774132    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75j9w-wvz7c" event={"ID":"b55057b6-d0b0-4be8-a41c-a6e243e2b990","Type":"ContainerDied","Data":"8db7ee73cb0ccb95df69a139817c9ea598b71cbbd50be57e9da9001aeb6af65f"}
Oct 30 21:01:48.774503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.774160    4549 scope.go:122] "RemoveContainer" containerID="70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c"
Oct 30 21:01:48.782311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.782294    4549 scope.go:122] "RemoveContainer" containerID="70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c"
Oct 30 21:01:48.782583 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:01:48.782563    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c\": container with ID starting with 70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c not found: ID does not exist" containerID="70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c"
Oct 30 21:01:48.782656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.782594    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c"} err="failed to get container status \"70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c\": rpc error: code = NotFound desc = could not find container \"70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c\": container with ID starting with 70672c4233df2d8af8f6f03f68f515661382ba6ae8a743f1848af9faafd0bf5c not found: ID does not exist"
Oct 30 21:01:48.822763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.822734    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-75j9w-wvz7c"]
Oct 30 21:01:48.834882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:48.834852    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-75j9w-wvz7c"]
Oct 30 21:01:49.341107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:01:49.341076    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b55057b6-d0b0-4be8-a41c-a6e243e2b990" path="/var/lib/kubelet/pods/b55057b6-d0b0-4be8-a41c-a6e243e2b990/volumes"
Oct 30 21:01:56.572276 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:01:56.572237    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:02:03.343383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:03.343351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.265670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:16.265634    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.385603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:16.385558    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:26.572621 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:02:26.572583    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:02:28.836936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:28.836854    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:28.955296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:28.955258    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:35.473724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:35.473684    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:35.596450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:35.596401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.535217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535180    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-79f47bdf55-s7dmf"]
Oct 30 21:02:56.535732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535542    4549 cpu_manager.go:414] "containerMap: removing container" podUID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" containerName="console"
Oct 30 21:02:56.535732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535558    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" containerName="console"
Oct 30 21:02:56.535732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535587    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b55057b6-d0b0-4be8-a41c-a6e243e2b990" containerName="insights-gathering"
Oct 30 21:02:56.535732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535596    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b55057b6-d0b0-4be8-a41c-a6e243e2b990" containerName="insights-gathering"
Oct 30 21:02:56.535732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535660    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b55057b6-d0b0-4be8-a41c-a6e243e2b990" containerName="insights-gathering"
Oct 30 21:02:56.535732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.535673    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9cebbd8-9bcb-42d6-98f3-6cfbd3b49def" containerName="console"
Oct 30 21:02:56.538657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.538636    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.574333 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:02:56.574291    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:02:56.576216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.576185    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79f47bdf55-s7dmf"]
Oct 30 21:02:56.661804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.661762    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed221151-cbc4-4d2f-be7e-b38e35e6f164-tmp\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.661804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.661808    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-trusted-ca-bundle\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.662096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.661909    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-oauth-config\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.662096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.661988    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-serving-cert\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.662096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.662047    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-oauth-serving-cert\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.662096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.662080    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-service-ca\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.662230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.662114    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qhzk\" (UniqueName: \"kubernetes.io/projected/ed221151-cbc4-4d2f-be7e-b38e35e6f164-kube-api-access-4qhzk\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.662230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.662156    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-config\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.691311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.691285    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.763203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763167    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-oauth-config\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763205    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-serving-cert\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763309    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-oauth-serving-cert\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763356    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-service-ca\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763378    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qhzk\" (UniqueName: \"kubernetes.io/projected/ed221151-cbc4-4d2f-be7e-b38e35e6f164-kube-api-access-4qhzk\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763418    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-config\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763455    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed221151-cbc4-4d2f-be7e-b38e35e6f164-tmp\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763772    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-trusted-ca-bundle\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.763945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.763924    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed221151-cbc4-4d2f-be7e-b38e35e6f164-tmp\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.764169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.764149    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-oauth-serving-cert\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.764487 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.764468    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-service-ca\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.764558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.764518    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-config\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.764980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.764953    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed221151-cbc4-4d2f-be7e-b38e35e6f164-trusted-ca-bundle\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.766112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.766092    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-oauth-config\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.766189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.766143    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed221151-cbc4-4d2f-be7e-b38e35e6f164-console-serving-cert\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.783923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.783889    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qhzk\" (UniqueName: \"kubernetes.io/projected/ed221151-cbc4-4d2f-be7e-b38e35e6f164-kube-api-access-4qhzk\") pod \"console-79f47bdf55-s7dmf\" (UID: \"ed221151-cbc4-4d2f-be7e-b38e35e6f164\") " pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.849699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.849660    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:02:56.895364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:56.895331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:57.020403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:57.020355    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:57.157004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:57.156925    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79f47bdf55-s7dmf"]
Oct 30 21:02:57.160024 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:02:57.159994    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded221151_cbc4_4d2f_be7e_b38e35e6f164.slice/crio-c78139b7f0630c94e1ae58dc5c21ff849aba52ea7f7e61211fcd86481faa7ee3 WatchSource:0}: Error finding container c78139b7f0630c94e1ae58dc5c21ff849aba52ea7f7e61211fcd86481faa7ee3: Status 404 returned error can't find the container with id c78139b7f0630c94e1ae58dc5c21ff849aba52ea7f7e61211fcd86481faa7ee3
Oct 30 21:02:58.003975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:58.003939    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f47bdf55-s7dmf" event={"ID":"ed221151-cbc4-4d2f-be7e-b38e35e6f164","Type":"ContainerStarted","Data":"14f046fc1c5e84c41f1cb20c04f60fe7351bdccf30d2e5433aae032864cfe292"}
Oct 30 21:02:58.004483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:58.003988    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f47bdf55-s7dmf" event={"ID":"ed221151-cbc4-4d2f-be7e-b38e35e6f164","Type":"ContainerStarted","Data":"c78139b7f0630c94e1ae58dc5c21ff849aba52ea7f7e61211fcd86481faa7ee3"}
Oct 30 21:02:58.045282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:02:58.045231    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-79f47bdf55-s7dmf" podStartSLOduration=2.045218227 podStartE2EDuration="2.045218227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:58.04439531 +0000 UTC m=+160.818795261" watchObservedRunningTime="2026-10-30 21:02:58.045218227 +0000 UTC m=+160.819618167"
Oct 30 21:03:03.473374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:03.473341    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:06.850137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:06.850100    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:03:06.850604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:06.850171    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:03:06.854718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:06.854695    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:03:07.035514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:07.035487    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79f47bdf55-s7dmf"
Oct 30 21:03:07.110698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:07.110611    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bcf94998-qlwcl"]
Oct 30 21:03:26.572265 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:03:26.572230    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:03:32.130720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.130660    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-67bcf94998-qlwcl" podUID="e341d71e-73c8-4f60-a143-09f2267d0869" containerName="console" containerID="cri-o://2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b" gracePeriod=15
Oct 30 21:03:32.373924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.373895    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:03:32.437454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437355    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.437454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437448    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.437678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437477    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.437678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437502    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.437789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437751    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:03:32.437905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437880    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:32.437905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437892    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:32.437974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.437918    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:32.537937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.537895    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.537937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.537937    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.538168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.537967    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.538168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.538046    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config\") pod \"e341d71e-73c8-4f60-a143-09f2267d0869\" (UID: \"e341d71e-73c8-4f60-a143-09f2267d0869\") "
Oct 30 21:03:32.538275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.538231    4549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-oauth-serving-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.538275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.538250    4549 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e341d71e-73c8-4f60-a143-09f2267d0869-tmp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.538275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.538264    4549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-console-config\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.538428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.538279    4549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-trusted-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.538428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.538313    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:32.540222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.540191    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:03:32.540340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.540254    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:03:32.540340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.540256    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l" pod "e341d71e-73c8-4f60-a143-09f2267d0869" (UID: "e341d71e-73c8-4f60-a143-09f2267d0869"). InnerVolumeSpecName "kube-api-access-z7p5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:03:32.638857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.638791    4549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-oauth-config\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.638857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.638850    4549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e341d71e-73c8-4f60-a143-09f2267d0869-console-serving-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.638857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.638865    4549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e341d71e-73c8-4f60-a143-09f2267d0869-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:32.639110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:32.638883    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/e341d71e-73c8-4f60-a143-09f2267d0869-kube-api-access-z7p5l\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:03:33.118538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.118507    4549 generic.go:356] "Generic (PLEG): container finished" podID="e341d71e-73c8-4f60-a143-09f2267d0869" containerID="2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b" exitCode=0
Oct 30 21:03:33.118538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.118549    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcf94998-qlwcl" event={"ID":"e341d71e-73c8-4f60-a143-09f2267d0869","Type":"ContainerDied","Data":"2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b"}
Oct 30 21:03:33.118771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.118585    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcf94998-qlwcl" event={"ID":"e341d71e-73c8-4f60-a143-09f2267d0869","Type":"ContainerDied","Data":"ce451723d8df1f41da0fc1c5996f2e6f983646a0c28524be021b7f26f2b18d07"}
Oct 30 21:03:33.118771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.118605    4549 scope.go:122] "RemoveContainer" containerID="2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b"
Oct 30 21:03:33.118771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.118552    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcf94998-qlwcl"
Oct 30 21:03:33.126987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.126964    4549 scope.go:122] "RemoveContainer" containerID="2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b"
Oct 30 21:03:33.127282 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:03:33.127261    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b\": container with ID starting with 2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b not found: ID does not exist" containerID="2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b"
Oct 30 21:03:33.127389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.127292    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b"} err="failed to get container status \"2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b\": rpc error: code = NotFound desc = could not find container \"2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b\": container with ID starting with 2f65d6c8347fd122aed70b26d0eb581532be56663c5da8e3b498fecb269f1e9b not found: ID does not exist"
Oct 30 21:03:33.167260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.167230    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bcf94998-qlwcl"]
Oct 30 21:03:33.179143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.179105    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67bcf94998-qlwcl"]
Oct 30 21:03:33.340461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:33.340427    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e341d71e-73c8-4f60-a143-09f2267d0869" path="/var/lib/kubelet/pods/e341d71e-73c8-4f60-a143-09f2267d0869/volumes"
Oct 30 21:03:55.323983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:55.323951    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:55.501479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:55.501442    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:55.653018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:55.652914    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.573498 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:03:56.573463    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:03:56.894644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.894555    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"]
Oct 30 21:03:56.894821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.894802    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e341d71e-73c8-4f60-a143-09f2267d0869" containerName="console"
Oct 30 21:03:56.894821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.894812    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e341d71e-73c8-4f60-a143-09f2267d0869" containerName="console"
Oct 30 21:03:56.894974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.894894    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e341d71e-73c8-4f60-a143-09f2267d0869" containerName="console"
Oct 30 21:03:56.899290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.899266    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:56.914219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.914191    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:03:56.914219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.914191    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:03:56.914434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.914420    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wmcm5\""
Oct 30 21:03:56.914434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.914428    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:03:56.926575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.926544    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:03:56.926751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.926597    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:03:56.926751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.926678    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:03:56.927602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.927579    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"]
Oct 30 21:03:56.940355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:56.940326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:57.009279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009235    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.009279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009280    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.009520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009394    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.009520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009432    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.009520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009448    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.009520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009472    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.009709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.009584    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-669r5\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.069046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.069012    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:57.110947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.110914    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.110952    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-669r5\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.111154    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.111185    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.111243    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.111275    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.111301    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.111961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.111881    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.113600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.113573    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.113718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.113652    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.113718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.113670    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.113870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.113822    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.131040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.131007    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.131253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.131231    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-669r5\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5\") pod \"istiod-openshift-gateway-7876bfcb74-47kpq\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.210505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.210415    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:03:57.522762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:57.522691    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"]
Oct 30 21:03:57.526074 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:03:57.526041    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb469d77c_90bc_48e7_9363_37cb86955f99.slice/crio-95f3152a6e8830093f394b3af0924dbdd0ade6d0c1cf7acb791868680a5945a7 WatchSource:0}: Error finding container 95f3152a6e8830093f394b3af0924dbdd0ade6d0c1cf7acb791868680a5945a7: Status 404 returned error can't find the container with id 95f3152a6e8830093f394b3af0924dbdd0ade6d0c1cf7acb791868680a5945a7
Oct 30 21:03:58.020217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:58.020175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:58.203152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:58.203112    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" event={"ID":"b469d77c-90bc-48e7-9363-37cb86955f99","Type":"ContainerStarted","Data":"95f3152a6e8830093f394b3af0924dbdd0ade6d0c1cf7acb791868680a5945a7"}
Oct 30 21:03:59.909614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:59.909579    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:03:59.909926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:03:59.909647    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:04:00.211967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.211887    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" event={"ID":"b469d77c-90bc-48e7-9363-37cb86955f99","Type":"ContainerStarted","Data":"88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f"}
Oct 30 21:04:00.212122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.212101    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:04:00.213846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.213805    4549 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-47kpq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:04:00.213962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.213870    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:04:00.270124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.270070    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" podStartSLOduration=1.8893378109999999 podStartE2EDuration="4.27005654s" totalImagesPullingTime="2.380718729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:04:00.264251704 +0000 UTC m=+223.038651645" watchObservedRunningTime="2026-10-30 21:04:00.27005654 +0000 UTC m=+223.044456480"
Oct 30 21:04:00.620326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.620293    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:00.764621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:00.764574    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:01.216244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:01.216203    4549 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-47kpq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:04:01.216632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:01.216276    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:04:01.576277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:01.576240    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:01.706712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:01.706681    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:01.832377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:01.832289    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:03.588722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:03.588688    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:04.216716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:04.216688    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:04:05.030367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:05.030334    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:05.980613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:05.980578    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:15.890141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:15.890055    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:16.016451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:16.016414    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:16.396417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:16.396384    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:16.529963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:16.529932    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:20.267354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:20.267308    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:22.916130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:22.916096    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:23.042769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:23.042741    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:24.386493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:24.386459    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:26.572257 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:04:26.572209    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:04:34.076224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:34.076189    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:34.220192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:34.220149    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.005278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.005247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.154150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.154107    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"]
Oct 30 21:04:35.159298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.159272    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.203565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.203531    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"]
Oct 30 21:04:35.267985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.267872    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:04:35.268143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.268033    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-t56h2\""
Oct 30 21:04:35.268143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.268036    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:04:35.268143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.268118    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:04:35.271864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.271842    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:04:35.271951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.271936    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:04:35.307994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.307956    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-metrics-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.308088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.308003    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.308088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.308030    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-webhook-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.308175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.308155    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-tmp\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.308209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.308191    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr2fj\" (UniqueName: \"kubernetes.io/projected/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-kube-api-access-wr2fj\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.308281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.308265    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-manifests\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.321750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.321714    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.408987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.408945    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-tmp\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.408987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.408984    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr2fj\" (UniqueName: \"kubernetes.io/projected/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-kube-api-access-wr2fj\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.409242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.409009    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-manifests\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.409242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.409043    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-metrics-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.409242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.409059    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.409242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.409081    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-webhook-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.409468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.409447    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-manifests\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.411492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.411461    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-tmp\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.411625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.411508    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-webhook-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.411625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.411612    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-metrics-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.411733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.411660    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.431924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.431895    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr2fj\" (UniqueName: \"kubernetes.io/projected/d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2-kube-api-access-wr2fj\") pod \"opendatahub-operator-controller-manager-98586cfbd-dzcmv\" (UID: \"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.448807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.448769    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.471802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.471773    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:35.601481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.601449    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.815822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:35.815791    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"]
Oct 30 21:04:35.830067 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:04:35.830038    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4bb8604_4f63_485a_9a3f_2d68ba1bcaf2.slice/crio-790ec5c82a2b364f69c8772969a3de0b56f4e532446f1b75c004b3058f79bcc7 WatchSource:0}: Error finding container 790ec5c82a2b364f69c8772969a3de0b56f4e532446f1b75c004b3058f79bcc7: Status 404 returned error can't find the container with id 790ec5c82a2b364f69c8772969a3de0b56f4e532446f1b75c004b3058f79bcc7
Oct 30 21:04:36.334421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:36.334376    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv" event={"ID":"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2","Type":"ContainerStarted","Data":"790ec5c82a2b364f69c8772969a3de0b56f4e532446f1b75c004b3058f79bcc7"}
Oct 30 21:04:38.078243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:38.078207    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:38.737730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:38.737691    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:38.957859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:38.957724    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:39.346538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:39.346502    4549 generic.go:356] "Generic (PLEG): container finished" podID="d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2" containerID="c90aba6301096e8cc6e2db6563b5b9ed117b8128acd864cd99ec522121b86a8f" exitCode=0
Oct 30 21:04:39.346962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:39.346584    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv" event={"ID":"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2","Type":"ContainerDied","Data":"c90aba6301096e8cc6e2db6563b5b9ed117b8128acd864cd99ec522121b86a8f"}
Oct 30 21:04:40.351477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:40.351447    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv" event={"ID":"d4bb8604-4f63-485a-9a3f-2d68ba1bcaf2","Type":"ContainerStarted","Data":"c7176a65c22e344c3b40aa461b1c33cfa59540b5fa652672aedd953671493026"}
Oct 30 21:04:40.351890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:40.351583    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:40.441406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:40.441089    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv" podStartSLOduration=1.911111647 podStartE2EDuration="5.441068992s" totalImagesPullingTime="3.102749595s" totalInitContainerRuntime="427.20775ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:04:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:04:40.427326392 +0000 UTC m=+263.201726333" watchObservedRunningTime="2026-10-30 21:04:40.441068992 +0000 UTC m=+263.215468934"
Oct 30 21:04:40.781923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:40.781822    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:43.287776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:43.287740    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:43.430507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:43.430472    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:43.658060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:43.658018    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:45.834021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:45.833987    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:46.012978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:46.012941    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:48.295781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:48.295744    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:48.451508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:48.451467    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:51.357436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:51.357410    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-98586cfbd-dzcmv"
Oct 30 21:04:52.476951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:52.476916    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:53.094779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:53.094745    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:53.312788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:53.312746    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:53.477492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:53.477411    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:55.388940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:55.388906    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:56.572082 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:04:56.572043    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:04:56.871239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:56.871155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:57.291040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:57.290953    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:57.427022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:57.426974    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:57.653956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:57.653924    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:58.648759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:58.648723    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:59.889974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:04:59.889941    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:02.709582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:02.709549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:02.842017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:02.841985    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:03.725817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:03.725782    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:04.058407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:04.058312    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:05.709941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:05.709888    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:07.330215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:07.330182    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:07.509125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:07.509093    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.067017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.066986    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.194250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.194210    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.471511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.471433    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.633465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.633431    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"]
Oct 30 21:05:08.636978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.636952    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.643514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.643482    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-tls-certs\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.643674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.643523    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-tmp\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.643674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.643637    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5qqb\" (UniqueName: \"kubernetes.io/projected/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-kube-api-access-j5qqb\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.663980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.663940    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.670427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.670395    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"]
Oct 30 21:05:08.692224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.692190    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:05:08.692432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.692268    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:05:08.692432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.692366    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6kfs6\""
Oct 30 21:05:08.744318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.744227    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-tls-certs\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.744318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.744262    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-tmp\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.744318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.744294    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qqb\" (UniqueName: \"kubernetes.io/projected/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-kube-api-access-j5qqb\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.746895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.746868    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-tmp\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.747026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.747008    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-tls-certs\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.770801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.770758    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5qqb\" (UniqueName: \"kubernetes.io/projected/d69f0413-b858-4239-8aeb-5b2e6b27b1d6-kube-api-access-j5qqb\") pod \"kube-auth-proxy-5649f7bc66-klvr2\" (UID: \"d69f0413-b858-4239-8aeb-5b2e6b27b1d6\") " pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.830581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.830546    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.951689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.951642    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"
Oct 30 21:05:08.955964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:08.955934    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:09.211275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:09.211241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:09.251810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:09.251777    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2"]
Oct 30 21:05:09.255259 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:05:09.255232    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69f0413_b858_4239_8aeb_5b2e6b27b1d6.slice/crio-ba3c8ae10668ddfecc1f774c76010f9c1eeddf178d12dd52f9cd84f098cac22f WatchSource:0}: Error finding container ba3c8ae10668ddfecc1f774c76010f9c1eeddf178d12dd52f9cd84f098cac22f: Status 404 returned error can't find the container with id ba3c8ae10668ddfecc1f774c76010f9c1eeddf178d12dd52f9cd84f098cac22f
Oct 30 21:05:09.452369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:09.452325    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2" event={"ID":"d69f0413-b858-4239-8aeb-5b2e6b27b1d6","Type":"ContainerStarted","Data":"ba3c8ae10668ddfecc1f774c76010f9c1eeddf178d12dd52f9cd84f098cac22f"}
Oct 30 21:05:09.788478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:09.788376    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:09.915303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:09.915263    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:10.054347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:10.054262    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:10.579275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:10.579224    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:10.846647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:10.846538    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:11.016076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:11.016041    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:11.555511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:11.555429    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.104324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:12.104291    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.293785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:12.293748    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.465927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:12.465853    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2" event={"ID":"d69f0413-b858-4239-8aeb-5b2e6b27b1d6","Type":"ContainerStarted","Data":"282220fa0fa76cd9107402fd940085a89c54774290b6aa5f31c67ccfd4013d09"}
Oct 30 21:05:12.489103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:12.489076    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.517075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:12.517023    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2" podStartSLOduration=1.981590045 podStartE2EDuration="4.517009751s" totalImagesPullingTime="2.535419706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:05:12.516512629 +0000 UTC m=+295.290912570" watchObservedRunningTime="2026-10-30 21:05:12.517009751 +0000 UTC m=+295.291409716"
Oct 30 21:05:15.105286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:15.105248    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:15.325682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:15.325644    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:17.358258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:17.358232    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:05:17.358679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:17.358413    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:05:17.516589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:17.516549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:17.667283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:17.667194    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:17.804689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:17.804656    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:22.223892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:22.223857    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:22.478296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:22.478214    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:24.696066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:24.696033    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:24.938761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:24.938723    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:26.571719 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:05:26.571677    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:05:27.168765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:27.168731    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:27.322218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:27.322185    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:32.559430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:32.559343    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:32.709482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:32.709438    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:32.953295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:32.953262    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:33.097076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:33.097044    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:37.230545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:37.230510    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:37.353435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:37.353404    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:44.346518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:44.346477    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:44.486649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:44.486616    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:48.247060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.247026    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:05:48.254961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.254934    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.259227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.259199    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:48.270778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.270753    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:05:48.281804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.281781    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:05:48.281804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.281782    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:05:48.283069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.283051    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lmssf\""
Oct 30 21:05:48.283123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.283053    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:05:48.356116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.356074    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8d24\" (UniqueName: \"kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24\") pod \"limitador-limitador-74dfcfb894-xmjhb\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.356359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.356140    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file\") pod \"limitador-limitador-74dfcfb894-xmjhb\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.375425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.375388    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:05:48.456880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.456816    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8d24\" (UniqueName: \"kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24\") pod \"limitador-limitador-74dfcfb894-xmjhb\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.457086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.456935    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file\") pod \"limitador-limitador-74dfcfb894-xmjhb\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.457610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.457586    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file\") pod \"limitador-limitador-74dfcfb894-xmjhb\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.476275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.476250    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8d24\" (UniqueName: \"kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24\") pod \"limitador-limitador-74dfcfb894-xmjhb\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.568343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.568308    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:48.862188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.862113    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:05:48.865173 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:05:48.865142    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842342c4_dc97_4b09_baa7_3e712773636c.slice/crio-700b8a38cbc1ba535281cd554a0cc10acc64592bbbeeb66015117f5e3c2f141f WatchSource:0}: Error finding container 700b8a38cbc1ba535281cd554a0cc10acc64592bbbeeb66015117f5e3c2f141f: Status 404 returned error can't find the container with id 700b8a38cbc1ba535281cd554a0cc10acc64592bbbeeb66015117f5e3c2f141f
Oct 30 21:05:48.866988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:48.866972    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:05:49.597953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:49.597919    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb" event={"ID":"842342c4-dc97-4b09-baa7-3e712773636c","Type":"ContainerStarted","Data":"700b8a38cbc1ba535281cd554a0cc10acc64592bbbeeb66015117f5e3c2f141f"}
Oct 30 21:05:51.607270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:51.607235    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb" event={"ID":"842342c4-dc97-4b09-baa7-3e712773636c","Type":"ContainerStarted","Data":"7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32"}
Oct 30 21:05:51.607692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:51.607334    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:05:51.651183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:05:51.651077    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb" podStartSLOduration=1.138562106 podStartE2EDuration="3.65106381s" totalImagesPullingTime="2.512501704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:05:51.65037082 +0000 UTC m=+334.424770951" watchObservedRunningTime="2026-10-30 21:05:51.65106381 +0000 UTC m=+334.425463745"
Oct 30 21:05:56.571259 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:05:56.571221    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:06:02.615489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:02.615447    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:06:02.967421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:02.967323    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:06:02.967608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:02.967550    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb" podUID="842342c4-dc97-4b09-baa7-3e712773636c" containerName="limitador" containerID="cri-o://7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32" gracePeriod=30
Oct 30 21:06:03.529953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.529785    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:06:03.654767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.654735    4549 generic.go:356] "Generic (PLEG): container finished" podID="842342c4-dc97-4b09-baa7-3e712773636c" containerID="7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32" exitCode=0
Oct 30 21:06:03.655354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.654779    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb" event={"ID":"842342c4-dc97-4b09-baa7-3e712773636c","Type":"ContainerDied","Data":"7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32"}
Oct 30 21:06:03.655354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.654787    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"
Oct 30 21:06:03.655354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.654822    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xmjhb" event={"ID":"842342c4-dc97-4b09-baa7-3e712773636c","Type":"ContainerDied","Data":"700b8a38cbc1ba535281cd554a0cc10acc64592bbbeeb66015117f5e3c2f141f"}
Oct 30 21:06:03.655354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.654868    4549 scope.go:122] "RemoveContainer" containerID="7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32"
Oct 30 21:06:03.663075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.663054    4549 scope.go:122] "RemoveContainer" containerID="7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32"
Oct 30 21:06:03.663427 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:06:03.663404    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32\": container with ID starting with 7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32 not found: ID does not exist" containerID="7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32"
Oct 30 21:06:03.663507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.663442    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32"} err="failed to get container status \"7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32\": rpc error: code = NotFound desc = could not find container \"7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32\": container with ID starting with 7c85e7c80f32fbb51bff068d3fd1d82c68237d14b33ee5025a4b4c86f185cc32 not found: ID does not exist"
Oct 30 21:06:03.686896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.686860    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file\" (UniqueName: \"kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file\") pod \"842342c4-dc97-4b09-baa7-3e712773636c\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") "
Oct 30 21:06:03.687079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.686917    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24\" (UniqueName: \"kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24\") pod \"842342c4-dc97-4b09-baa7-3e712773636c\" (UID: \"842342c4-dc97-4b09-baa7-3e712773636c\") "
Oct 30 21:06:03.687233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.687211    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file" pod "842342c4-dc97-4b09-baa7-3e712773636c" (UID: "842342c4-dc97-4b09-baa7-3e712773636c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:06:03.689325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.689293    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24" pod "842342c4-dc97-4b09-baa7-3e712773636c" (UID: "842342c4-dc97-4b09-baa7-3e712773636c"). InnerVolumeSpecName "kube-api-access-n8d24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:06:03.788302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.788206    4549 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/842342c4-dc97-4b09-baa7-3e712773636c-config-file\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:03.788302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.788241    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8d24\" (UniqueName: \"kubernetes.io/projected/842342c4-dc97-4b09-baa7-3e712773636c-kube-api-access-n8d24\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:03.791248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.791220    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:03.997932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:03.997896    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:06:04.009847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:04.009793    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xmjhb"]
Oct 30 21:06:05.341949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:05.341919    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="842342c4-dc97-4b09-baa7-3e712773636c" path="/var/lib/kubelet/pods/842342c4-dc97-4b09-baa7-3e712773636c/volumes"
Oct 30 21:06:06.907344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:06.907308    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:07.128887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.128850    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:07.630326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.630296    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"]
Oct 30 21:06:07.630578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.630566    4549 cpu_manager.go:414] "containerMap: removing container" podUID="842342c4-dc97-4b09-baa7-3e712773636c" containerName="limitador"
Oct 30 21:06:07.630620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.630579    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="842342c4-dc97-4b09-baa7-3e712773636c" containerName="limitador"
Oct 30 21:06:07.630654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.630643    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="842342c4-dc97-4b09-baa7-3e712773636c" containerName="limitador"
Oct 30 21:06:07.635030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.635009    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.655794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.655751    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:06:07.667634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.667595    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:06:07.667793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.667704    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cwdwz\""
Oct 30 21:06:07.667793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.667753    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 21:06:07.669181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.669151    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"]
Oct 30 21:06:07.721069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.721030    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d92ef50b-4758-42be-9b06-21b319ec61a8-data\") pod \"postgres-7cdbf6dc44-9qhzp\" (UID: \"d92ef50b-4758-42be-9b06-21b319ec61a8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.721246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.721078    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vv8b\" (UniqueName: \"kubernetes.io/projected/d92ef50b-4758-42be-9b06-21b319ec61a8-kube-api-access-7vv8b\") pod \"postgres-7cdbf6dc44-9qhzp\" (UID: \"d92ef50b-4758-42be-9b06-21b319ec61a8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.763028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.762995    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:07.822183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.822145    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d92ef50b-4758-42be-9b06-21b319ec61a8-data\") pod \"postgres-7cdbf6dc44-9qhzp\" (UID: \"d92ef50b-4758-42be-9b06-21b319ec61a8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.822368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.822203    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vv8b\" (UniqueName: \"kubernetes.io/projected/d92ef50b-4758-42be-9b06-21b319ec61a8-kube-api-access-7vv8b\") pod \"postgres-7cdbf6dc44-9qhzp\" (UID: \"d92ef50b-4758-42be-9b06-21b319ec61a8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.822560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.822540    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d92ef50b-4758-42be-9b06-21b319ec61a8-data\") pod \"postgres-7cdbf6dc44-9qhzp\" (UID: \"d92ef50b-4758-42be-9b06-21b319ec61a8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.842162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.842125    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vv8b\" (UniqueName: \"kubernetes.io/projected/d92ef50b-4758-42be-9b06-21b319ec61a8-kube-api-access-7vv8b\") pod \"postgres-7cdbf6dc44-9qhzp\" (UID: \"d92ef50b-4758-42be-9b06-21b319ec61a8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:07.947974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:07.947880    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:08.260413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:08.260332    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"]
Oct 30 21:06:08.263966 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:06:08.263938    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92ef50b_4758_42be_9b06_21b319ec61a8.slice/crio-76c79cced7c7d80a89221de20917fc99e8a8f1888039b1e294553caf355d3bfe WatchSource:0}: Error finding container 76c79cced7c7d80a89221de20917fc99e8a8f1888039b1e294553caf355d3bfe: Status 404 returned error can't find the container with id 76c79cced7c7d80a89221de20917fc99e8a8f1888039b1e294553caf355d3bfe
Oct 30 21:06:08.683078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:08.683041    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp" event={"ID":"d92ef50b-4758-42be-9b06-21b319ec61a8","Type":"ContainerStarted","Data":"76c79cced7c7d80a89221de20917fc99e8a8f1888039b1e294553caf355d3bfe"}
Oct 30 21:06:11.937695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.937666    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"]
Oct 30 21:06:11.941101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.941078    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.972076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.972046    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"]
Oct 30 21:06:11.977206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977170    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.977362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977227    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2272918-faed-4465-b14f-19a53468a454-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.977362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977261    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.977362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977321    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2272918-faed-4465-b14f-19a53468a454-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.977362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977345    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.977573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977392    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvdjl\" (UniqueName: \"kubernetes.io/projected/a2272918-faed-4465-b14f-19a53468a454-kube-api-access-bvdjl\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:11.977573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:11.977425    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a2272918-faed-4465-b14f-19a53468a454-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078177    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvdjl\" (UniqueName: \"kubernetes.io/projected/a2272918-faed-4465-b14f-19a53468a454-kube-api-access-bvdjl\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078216    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a2272918-faed-4465-b14f-19a53468a454-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078267    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078303    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2272918-faed-4465-b14f-19a53468a454-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078335    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078389    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2272918-faed-4465-b14f-19a53468a454-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.078419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.078411    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.079075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.079053    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a2272918-faed-4465-b14f-19a53468a454-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.082069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.082024    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2272918-faed-4465-b14f-19a53468a454-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.082184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.082103    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.082360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.082339    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.084411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.084363    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a2272918-faed-4465-b14f-19a53468a454-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.098640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.098611    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2272918-faed-4465-b14f-19a53468a454-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.099427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.099402    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvdjl\" (UniqueName: \"kubernetes.io/projected/a2272918-faed-4465-b14f-19a53468a454-kube-api-access-bvdjl\") pod \"istiod-openshift-gateway-7ddd9cfdc8-jggkz\" (UID: \"a2272918-faed-4465-b14f-19a53468a454\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.257320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.257234    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.431404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.431366    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:12.565401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.565363    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:12.587353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.587323    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"]
Oct 30 21:06:12.599893 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:06:12.599859    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2272918_faed_4465_b14f_19a53468a454.slice/crio-7b791cc1e06e75823eeebca6ebc4b0f17cb1ed31ea3908a196488d79dadff6b9 WatchSource:0}: Error finding container 7b791cc1e06e75823eeebca6ebc4b0f17cb1ed31ea3908a196488d79dadff6b9: Status 404 returned error can't find the container with id 7b791cc1e06e75823eeebca6ebc4b0f17cb1ed31ea3908a196488d79dadff6b9
Oct 30 21:06:12.602449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.602412    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:06:12.602578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.602493    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:06:12.702894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.702859    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz" event={"ID":"a2272918-faed-4465-b14f-19a53468a454","Type":"ContainerStarted","Data":"d3242908d9e31c3463d0ded483cad8c032a2e25b8568c234bd66a5c3ed103832"}
Oct 30 21:06:12.703059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.702920    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz" event={"ID":"a2272918-faed-4465-b14f-19a53468a454","Type":"ContainerStarted","Data":"7b791cc1e06e75823eeebca6ebc4b0f17cb1ed31ea3908a196488d79dadff6b9"}
Oct 30 21:06:12.703691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.703673    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:12.715296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.715264    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:12.750989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.750937    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz" podStartSLOduration=1.750918123 podStartE2EDuration="1.750918123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:12.748907648 +0000 UTC m=+355.523307586" watchObservedRunningTime="2026-10-30 21:06:12.750918123 +0000 UTC m=+355.525318068"
Oct 30 21:06:12.859158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.859078    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:12.996253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:12.996215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:13.146716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:13.146599    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:13.709676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:13.709630    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp" event={"ID":"d92ef50b-4758-42be-9b06-21b319ec61a8","Type":"ContainerStarted","Data":"cbf3faad1eb2da55150a838325b0241d0173066a809ddf30c9c4f4b6eccf4ddb"}
Oct 30 21:06:13.711720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:13.711698    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz"
Oct 30 21:06:13.748442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:13.748394    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp" podStartSLOduration=1.723088657 podStartE2EDuration="6.748380183s" totalImagesPullingTime="5.025291526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:13.747865004 +0000 UTC m=+356.522264939" watchObservedRunningTime="2026-10-30 21:06:13.748380183 +0000 UTC m=+356.522780122"
Oct 30 21:06:13.819511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:13.819479    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"]
Oct 30 21:06:13.820235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:13.820203    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" containerName="discovery" containerID="cri-o://88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f" gracePeriod=30
Oct 30 21:06:14.064329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.064296    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:06:14.093701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093671    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.093903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093725    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.093903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093747    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.093903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093781    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.093903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093800    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.093903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093859    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.093903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.093890    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token\") pod \"b469d77c-90bc-48e7-9363-37cb86955f99\" (UID: \"b469d77c-90bc-48e7-9363-37cb86955f99\") "
Oct 30 21:06:14.094227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.094136    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:06:14.097075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.097038    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:06:14.102605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.097090    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:06:14.102933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.102609    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:06:14.103064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.102974    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "kube-api-access-669r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:06:14.103064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.102986    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:06:14.103507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.103364    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts" pod "b469d77c-90bc-48e7-9363-37cb86955f99" (UID: "b469d77c-90bc-48e7-9363-37cb86955f99"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:06:14.194360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194325    4549 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-cacerts\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.194360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194357    4549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-dns-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.194360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194369    4549 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b469d77c-90bc-48e7-9363-37cb86955f99-istio-kubeconfig\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.194360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194378    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.194633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194388    4549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b469d77c-90bc-48e7-9363-37cb86955f99-istio-csr-ca-configmap\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.194633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194396    4549 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b469d77c-90bc-48e7-9363-37cb86955f99-local-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.194633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.194404    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-669r5\" (UniqueName: \"kubernetes.io/projected/b469d77c-90bc-48e7-9363-37cb86955f99-kube-api-access-669r5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:14.717821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.717787    4549 generic.go:356] "Generic (PLEG): container finished" podID="b469d77c-90bc-48e7-9363-37cb86955f99" containerID="88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f" exitCode=0
Oct 30 21:06:14.718008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.717850    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"
Oct 30 21:06:14.718008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.717876    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" event={"ID":"b469d77c-90bc-48e7-9363-37cb86955f99","Type":"ContainerDied","Data":"88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f"}
Oct 30 21:06:14.718008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.717937    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq" event={"ID":"b469d77c-90bc-48e7-9363-37cb86955f99","Type":"ContainerDied","Data":"95f3152a6e8830093f394b3af0924dbdd0ade6d0c1cf7acb791868680a5945a7"}
Oct 30 21:06:14.718008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.717971    4549 scope.go:122] "RemoveContainer" containerID="88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f"
Oct 30 21:06:14.718312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.718294    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:14.727105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.727084    4549 scope.go:122] "RemoveContainer" containerID="88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f"
Oct 30 21:06:14.727424 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:06:14.727403    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f\": container with ID starting with 88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f not found: ID does not exist" containerID="88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f"
Oct 30 21:06:14.727490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.727434    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f"} err="failed to get container status \"88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f\": rpc error: code = NotFound desc = could not find container \"88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f\": container with ID starting with 88bc81404e2b33bef5d6f6889eb30fa4040e0987af3b859b5bf4b2a98510567f not found: ID does not exist"
Oct 30 21:06:14.781647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.781608    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"]
Oct 30 21:06:14.786237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.786205    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq"]
Oct 30 21:06:14.961143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:14.961106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:15.124069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:15.124039    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:15.341218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:15.341187    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" path="/var/lib/kubelet/pods/b469d77c-90bc-48e7-9363-37cb86955f99/volumes"
Oct 30 21:06:15.424204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:15.424122    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:20.752024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:20.751996    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9qhzp"
Oct 30 21:06:26.573499 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:06:26.573459    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:06:27.981044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:27.981012    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:28.021864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.021805    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-tzzrc"]
Oct 30 21:06:28.022231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.022209    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" containerName="discovery"
Oct 30 21:06:28.022334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.022233    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" containerName="discovery"
Oct 30 21:06:28.022431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.022361    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b469d77c-90bc-48e7-9363-37cb86955f99" containerName="discovery"
Oct 30 21:06:28.049578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.049536    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.054526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.054497    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-tzzrc"]
Oct 30 21:06:28.079266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.079230    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:06:28.079511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.079382    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:06:28.079511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.079435    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:06:28.079511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.079447    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:06:28.079511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.079481    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mhc8s\""
Oct 30 21:06:28.079511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.079493    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:06:28.101423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.101379    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.101423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.101424    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.101639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.101456    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfpjs\" (UniqueName: \"kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.135884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.135822    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:28.202196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.202155    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.202196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.202196    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.202405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.202229    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfpjs\" (UniqueName: \"kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.204942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.204918    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.205042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.204948    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.222244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.222212    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfpjs\" (UniqueName: \"kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs\") pod \"maas-controller-579cf7c6d-tzzrc\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") " pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.287944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.287824    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:28.362745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.362707    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:28.388619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.388588    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-654c5dd44b-q8pnp"]
Oct 30 21:06:28.393507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.393464    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.418915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.418245    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-q8pnp"]
Oct 30 21:06:28.505363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.505321    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.505363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.505371    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwgf8\" (UniqueName: \"kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.505590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.505479    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.606686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.606650    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.606870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.606709    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwgf8\" (UniqueName: \"kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.606870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.606774    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.612845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.610250    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.612845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.610317    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.628009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.627974    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwgf8\" (UniqueName: \"kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8\") pod \"maas-controller-654c5dd44b-q8pnp\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") " pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.667512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.667481    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-tzzrc"]
Oct 30 21:06:28.670266 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:06:28.670230    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5546581_5be3_41e6_9cc4_93eb180bb988.slice/crio-c49ec363e661fbb294494c07fc3ad43c3823b0461b737ba7b8beb8661d4de681 WatchSource:0}: Error finding container c49ec363e661fbb294494c07fc3ad43c3823b0461b737ba7b8beb8661d4de681: Status 404 returned error can't find the container with id c49ec363e661fbb294494c07fc3ad43c3823b0461b737ba7b8beb8661d4de681
Oct 30 21:06:28.708754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.708708    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:28.709783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.709752    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:28.772847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.772792    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-tzzrc" event={"ID":"c5546581-5be3-41e6-9cc4-93eb180bb988","Type":"ContainerStarted","Data":"c49ec363e661fbb294494c07fc3ad43c3823b0461b737ba7b8beb8661d4de681"}
Oct 30 21:06:28.865062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:28.864980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:29.000567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:29.000531    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-q8pnp"]
Oct 30 21:06:29.003701 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:06:29.003670    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b78b21f_3a4b_4988_84de_851f172a5d9f.slice/crio-4e2d2f2cc90c9457f2ca4bf5bbe704eac3fad75e8189f0ebd46b6d03996258ee WatchSource:0}: Error finding container 4e2d2f2cc90c9457f2ca4bf5bbe704eac3fad75e8189f0ebd46b6d03996258ee: Status 404 returned error can't find the container with id 4e2d2f2cc90c9457f2ca4bf5bbe704eac3fad75e8189f0ebd46b6d03996258ee
Oct 30 21:06:29.785226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:29.785185    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" event={"ID":"2b78b21f-3a4b-4988-84de-851f172a5d9f","Type":"ContainerStarted","Data":"4e2d2f2cc90c9457f2ca4bf5bbe704eac3fad75e8189f0ebd46b6d03996258ee"}
Oct 30 21:06:31.645245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:31.645204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:31.796189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:31.796157    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-tzzrc" event={"ID":"c5546581-5be3-41e6-9cc4-93eb180bb988","Type":"ContainerStarted","Data":"a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17"}
Oct 30 21:06:31.796360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:31.796283    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:31.798669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:31.798644    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:31.840988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:31.840939    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-tzzrc" podStartSLOduration=1.903175916 podStartE2EDuration="4.840925009s" totalImagesPullingTime="2.937749093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:31.840064806 +0000 UTC m=+374.614464736" watchObservedRunningTime="2026-10-30 21:06:31.840925009 +0000 UTC m=+374.615324948"
Oct 30 21:06:31.937785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:31.937695    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:32.801799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:32.801761    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" event={"ID":"2b78b21f-3a4b-4988-84de-851f172a5d9f","Type":"ContainerStarted","Data":"8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a"}
Oct 30 21:06:32.844230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:32.844178    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" podStartSLOduration=2.105014654 podStartE2EDuration="4.844164169s" totalImagesPullingTime="2.739149515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:32.842703178 +0000 UTC m=+375.617103117" watchObservedRunningTime="2026-10-30 21:06:32.844164169 +0000 UTC m=+375.618564107"
Oct 30 21:06:33.755686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.755651    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"]
Oct 30 21:06:33.759144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.759120    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.778652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.778618    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 21:06:33.791960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.790874    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-k2j9t\""
Oct 30 21:06:33.792550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.792528    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"]
Oct 30 21:06:33.796428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.796386    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 21:06:33.796547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.796431    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 21:06:33.815724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.815695    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:33.852496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.852464    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/74ff306c-2ab2-4ea5-8817-987e010342fc-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.852496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.852506    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ff306c-2ab2-4ea5-8817-987e010342fc-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.852699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.852525    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjzhg\" (UniqueName: \"kubernetes.io/projected/74ff306c-2ab2-4ea5-8817-987e010342fc-kube-api-access-kjzhg\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.852807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.852782    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/74ff306c-2ab2-4ea5-8817-987e010342fc-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.854587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.854562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:33.953973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.953936    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/74ff306c-2ab2-4ea5-8817-987e010342fc-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.953973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.953977    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ff306c-2ab2-4ea5-8817-987e010342fc-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.954201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.953995    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjzhg\" (UniqueName: \"kubernetes.io/projected/74ff306c-2ab2-4ea5-8817-987e010342fc-kube-api-access-kjzhg\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.954201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.954042    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/74ff306c-2ab2-4ea5-8817-987e010342fc-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.954649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.954626    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ff306c-2ab2-4ea5-8817-987e010342fc-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.954897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.954877    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/74ff306c-2ab2-4ea5-8817-987e010342fc-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.956539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.956519    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/74ff306c-2ab2-4ea5-8817-987e010342fc-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:33.974681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:33.974646    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjzhg\" (UniqueName: \"kubernetes.io/projected/74ff306c-2ab2-4ea5-8817-987e010342fc-kube-api-access-kjzhg\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-tmsm2\" (UID: \"74ff306c-2ab2-4ea5-8817-987e010342fc\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:34.035788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:34.035683    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:34.071879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:34.071811    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:34.366617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:34.366458    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"]
Oct 30 21:06:34.369325 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:06:34.369296    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ff306c_2ab2_4ea5_8817_987e010342fc.slice/crio-14e22391df82365856420d870eae9046e6eca2bcadaca9d64068d1105b62bbf0 WatchSource:0}: Error finding container 14e22391df82365856420d870eae9046e6eca2bcadaca9d64068d1105b62bbf0: Status 404 returned error can't find the container with id 14e22391df82365856420d870eae9046e6eca2bcadaca9d64068d1105b62bbf0
Oct 30 21:06:34.820687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:34.820647    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2" event={"ID":"74ff306c-2ab2-4ea5-8817-987e010342fc","Type":"ContainerStarted","Data":"14e22391df82365856420d870eae9046e6eca2bcadaca9d64068d1105b62bbf0"}
Oct 30 21:06:42.805081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:42.805045    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:44.823477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:44.823445    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:06:44.905060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:44.904986    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-tzzrc"]
Oct 30 21:06:44.905267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:44.905242    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-tzzrc" podUID="c5546581-5be3-41e6-9cc4-93eb180bb988" containerName="manager" containerID="cri-o://a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17" gracePeriod=30
Oct 30 21:06:45.166026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.166002    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:45.264723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.264691    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs\" (UniqueName: \"kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs\") pod \"c5546581-5be3-41e6-9cc4-93eb180bb988\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") "
Oct 30 21:06:45.264901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.264767    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert\") pod \"c5546581-5be3-41e6-9cc4-93eb180bb988\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") "
Oct 30 21:06:45.264960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.264909    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs\") pod \"c5546581-5be3-41e6-9cc4-93eb180bb988\" (UID: \"c5546581-5be3-41e6-9cc4-93eb180bb988\") "
Oct 30 21:06:45.267159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.267129    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs" pod "c5546581-5be3-41e6-9cc4-93eb180bb988" (UID: "c5546581-5be3-41e6-9cc4-93eb180bb988"). InnerVolumeSpecName "kube-api-access-zfpjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:06:45.267159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.267147    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert" pod "c5546581-5be3-41e6-9cc4-93eb180bb988" (UID: "c5546581-5be3-41e6-9cc4-93eb180bb988"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:06:45.267303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.267164    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs" pod "c5546581-5be3-41e6-9cc4-93eb180bb988" (UID: "c5546581-5be3-41e6-9cc4-93eb180bb988"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:06:45.366419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.366386    4549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-webhook-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:45.366419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.366411    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5546581-5be3-41e6-9cc4-93eb180bb988-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:45.366609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.366442    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfpjs\" (UniqueName: \"kubernetes.io/projected/c5546581-5be3-41e6-9cc4-93eb180bb988-kube-api-access-zfpjs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:06:45.873958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.873919    4549 generic.go:356] "Generic (PLEG): container finished" podID="c5546581-5be3-41e6-9cc4-93eb180bb988" containerID="a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17" exitCode=0
Oct 30 21:06:45.874437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.873980    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-tzzrc"
Oct 30 21:06:45.874437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.874002    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-tzzrc" event={"ID":"c5546581-5be3-41e6-9cc4-93eb180bb988","Type":"ContainerDied","Data":"a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17"}
Oct 30 21:06:45.874437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.874053    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-tzzrc" event={"ID":"c5546581-5be3-41e6-9cc4-93eb180bb988","Type":"ContainerDied","Data":"c49ec363e661fbb294494c07fc3ad43c3823b0461b737ba7b8beb8661d4de681"}
Oct 30 21:06:45.874437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.874080    4549 scope.go:122] "RemoveContainer" containerID="a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17"
Oct 30 21:06:45.875843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.875802    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2" event={"ID":"74ff306c-2ab2-4ea5-8817-987e010342fc","Type":"ContainerStarted","Data":"672873c25b3bd2ca126881c502ba08ab5eea95e82ea758a72d06799fd878f0a0"}
Oct 30 21:06:45.875969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.875880    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:06:45.882283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.882268    4549 scope.go:122] "RemoveContainer" containerID="a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17"
Oct 30 21:06:45.882554 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:06:45.882536    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17\": container with ID starting with a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17 not found: ID does not exist" containerID="a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17"
Oct 30 21:06:45.882599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.882562    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17"} err="failed to get container status \"a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17\": rpc error: code = NotFound desc = could not find container \"a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17\": container with ID starting with a16b83d5b48fff9ba53a5a9d7efa1af2400a0f43f464283616821baa9baeda17 not found: ID does not exist"
Oct 30 21:06:45.924512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.924482    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-tzzrc"]
Oct 30 21:06:45.936064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.936037    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-tzzrc"]
Oct 30 21:06:45.974312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:45.974257    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2" podStartSLOduration=2.302756966 podStartE2EDuration="12.974244271s" totalImagesPullingTime="10.671487305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:45.973733878 +0000 UTC m=+388.748133817" watchObservedRunningTime="2026-10-30 21:06:45.974244271 +0000 UTC m=+388.748644204"
Oct 30 21:06:47.341504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:47.341469    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5546581-5be3-41e6-9cc4-93eb180bb988" path="/var/lib/kubelet/pods/c5546581-5be3-41e6-9cc4-93eb180bb988/volumes"
Oct 30 21:06:56.572634 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:06:56.572598    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:06:56.886983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:06:56.886905    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tmsm2"
Oct 30 21:07:03.946252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:03.946161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:04.330357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:04.330317    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:04.561136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:04.561102    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:05.638374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.638342    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:05.783994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.783960    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:05.876233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.876199    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"]
Oct 30 21:07:05.876589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.876571    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c5546581-5be3-41e6-9cc4-93eb180bb988" containerName="manager"
Oct 30 21:07:05.876705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.876593    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5546581-5be3-41e6-9cc4-93eb180bb988" containerName="manager"
Oct 30 21:07:05.876705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.876668    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5546581-5be3-41e6-9cc4-93eb180bb988" containerName="manager"
Oct 30 21:07:05.880710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.880687    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:05.918957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.918891    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:07:05.919068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.918957    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4pdl4\""
Oct 30 21:07:05.919068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.919013    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:07:05.932764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.932739    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"]
Oct 30 21:07:05.936654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.936633    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:05.936770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.936690    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:05.936770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.936715    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:05.936913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.936868    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76bnf\" (UniqueName: \"kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:05.999102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:05.999064    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:06.038325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.038289    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.038473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.038335    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.038473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.038392    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76bnf\" (UniqueName: \"kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.038473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.038425    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.039154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.039087    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.040901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.040881    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.041048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.041031    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.062804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.062774    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76bnf\" (UniqueName: \"kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf\") pod \"maas-api-6d669c7fcd-nwf6b\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.191903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.191805    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:06.487239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.487129    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"]
Oct 30 21:07:06.489895 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:07:06.489869    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844c01d3_1113_4b80_a22d_2be606be8399.slice/crio-f50c2b2e4f5fcdf3436e1a9ef624c5af4d63a5fe3d49233c1122ada297263790 WatchSource:0}: Error finding container f50c2b2e4f5fcdf3436e1a9ef624c5af4d63a5fe3d49233c1122ada297263790: Status 404 returned error can't find the container with id f50c2b2e4f5fcdf3436e1a9ef624c5af4d63a5fe3d49233c1122ada297263790
Oct 30 21:07:06.960870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:06.960811    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b" event={"ID":"844c01d3-1113-4b80-a22d-2be606be8399","Type":"ContainerStarted","Data":"f50c2b2e4f5fcdf3436e1a9ef624c5af4d63a5fe3d49233c1122ada297263790"}
Oct 30 21:07:08.974082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:08.974034    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b" event={"ID":"844c01d3-1113-4b80-a22d-2be606be8399","Type":"ContainerStarted","Data":"d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5"}
Oct 30 21:07:08.974496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:08.974163    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:09.011979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:09.011924    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b" podStartSLOduration=2.477998748 podStartE2EDuration="4.011911735s" totalImagesPullingTime="1.533912987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:09.011799397 +0000 UTC m=+411.786199339" watchObservedRunningTime="2026-10-30 21:07:09.011911735 +0000 UTC m=+411.786311674"
Oct 30 21:07:14.983460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:14.983426    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:15.456030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:15.455992    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:15.625342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:15.625308    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:15.951849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:15.951800    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"]
Oct 30 21:07:16.120927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.120892    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"]
Oct 30 21:07:16.121349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.121019    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.227195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.227111    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.227195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.227146    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.227195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.227173    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkm6h\" (UniqueName: \"kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.227585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.227293    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.328362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.328318    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.328561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.328382    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.328561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.328404    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.328561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.328425    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkm6h\" (UniqueName: \"kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.329010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.328980    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.331030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.331009    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.331123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.331043    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.347662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.347630    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkm6h\" (UniqueName: \"kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h\") pod \"maas-api-5b898bf6b9-2hlbx\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") " pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.444203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.444170    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:16.807845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:16.807788    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"]
Oct 30 21:07:16.810613 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:07:16.810588    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b197df_f838_436e_9800_dab4962d7572.slice/crio-e0d3ca60ca7c698c10a7c67e3a4b8b5d9e0f7d27e2e98035945bbc73508185fb WatchSource:0}: Error finding container e0d3ca60ca7c698c10a7c67e3a4b8b5d9e0f7d27e2e98035945bbc73508185fb: Status 404 returned error can't find the container with id e0d3ca60ca7c698c10a7c67e3a4b8b5d9e0f7d27e2e98035945bbc73508185fb
Oct 30 21:07:17.006557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:17.006518    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" event={"ID":"47b197df-f838-436e-9800-dab4962d7572","Type":"ContainerStarted","Data":"e0d3ca60ca7c698c10a7c67e3a4b8b5d9e0f7d27e2e98035945bbc73508185fb"}
Oct 30 21:07:18.012286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:18.012249    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" event={"ID":"47b197df-f838-436e-9800-dab4962d7572","Type":"ContainerStarted","Data":"c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e"}
Oct 30 21:07:18.012719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:18.012392    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:18.054483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:18.054427    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" podStartSLOduration=2.524256489 podStartE2EDuration="3.054413579s" totalImagesPullingTime="530.15709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:18.052845936 +0000 UTC m=+420.827245869" watchObservedRunningTime="2026-10-30 21:07:18.054413579 +0000 UTC m=+420.828813556"
Oct 30 21:07:24.021997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.021962    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 21:07:24.088805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.088771    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"]
Oct 30 21:07:24.089048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.089010    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b" podUID="844c01d3-1113-4b80-a22d-2be606be8399" containerName="maas-api" containerID="cri-o://d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5" gracePeriod=30
Oct 30 21:07:24.330223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.330196    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:24.397813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.397781    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs\") pod \"844c01d3-1113-4b80-a22d-2be606be8399\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") "
Oct 30 21:07:24.398023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.397881    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls\") pod \"844c01d3-1113-4b80-a22d-2be606be8399\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") "
Oct 30 21:07:24.398023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.397925    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca\") pod \"844c01d3-1113-4b80-a22d-2be606be8399\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") "
Oct 30 21:07:24.398023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.397990    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf\" (UniqueName: \"kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf\") pod \"844c01d3-1113-4b80-a22d-2be606be8399\" (UID: \"844c01d3-1113-4b80-a22d-2be606be8399\") "
Oct 30 21:07:24.398320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.398293    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca" pod "844c01d3-1113-4b80-a22d-2be606be8399" (UID: "844c01d3-1113-4b80-a22d-2be606be8399"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:07:24.400219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.400193    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs" pod "844c01d3-1113-4b80-a22d-2be606be8399" (UID: "844c01d3-1113-4b80-a22d-2be606be8399"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:07:24.400219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.400197    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf" pod "844c01d3-1113-4b80-a22d-2be606be8399" (UID: "844c01d3-1113-4b80-a22d-2be606be8399"). InnerVolumeSpecName "kube-api-access-76bnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:07:24.400352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.400272    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls" pod "844c01d3-1113-4b80-a22d-2be606be8399" (UID: "844c01d3-1113-4b80-a22d-2be606be8399"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:07:24.499555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.499522    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76bnf\" (UniqueName: \"kubernetes.io/projected/844c01d3-1113-4b80-a22d-2be606be8399-kube-api-access-76bnf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:07:24.499555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.499552    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:07:24.499555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.499561    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/844c01d3-1113-4b80-a22d-2be606be8399-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:07:24.499797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:24.499570    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/844c01d3-1113-4b80-a22d-2be606be8399-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:07:25.040516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.040480    4549 generic.go:356] "Generic (PLEG): container finished" podID="844c01d3-1113-4b80-a22d-2be606be8399" containerID="d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5" exitCode=0
Oct 30 21:07:25.040935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.040528    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"
Oct 30 21:07:25.040935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.040540    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b" event={"ID":"844c01d3-1113-4b80-a22d-2be606be8399","Type":"ContainerDied","Data":"d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5"}
Oct 30 21:07:25.040935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.040584    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b" event={"ID":"844c01d3-1113-4b80-a22d-2be606be8399","Type":"ContainerDied","Data":"f50c2b2e4f5fcdf3436e1a9ef624c5af4d63a5fe3d49233c1122ada297263790"}
Oct 30 21:07:25.040935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.040605    4549 scope.go:122] "RemoveContainer" containerID="d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5"
Oct 30 21:07:25.049753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.049730    4549 scope.go:122] "RemoveContainer" containerID="d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5"
Oct 30 21:07:25.050065 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:07:25.050044    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5\": container with ID starting with d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5 not found: ID does not exist" containerID="d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5"
Oct 30 21:07:25.050124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.050076    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5"} err="failed to get container status \"d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5\": rpc error: code = NotFound desc = could not find container \"d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5\": container with ID starting with d4135285c2172098eaaafb06959797b5727d55aafad6ff97dde615ae20c790e5 not found: ID does not exist"
Oct 30 21:07:25.088473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.088437    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"]
Oct 30 21:07:25.100278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.100245    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-nwf6b"]
Oct 30 21:07:25.341589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:25.341554    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="844c01d3-1113-4b80-a22d-2be606be8399" path="/var/lib/kubelet/pods/844c01d3-1113-4b80-a22d-2be606be8399/volumes"
Oct 30 21:07:26.573325 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:07:26.573287    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:07:52.411625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:52.411588    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.567481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:07:52.567446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:56.572220 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:07:56.572179    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:08:04.062813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:04.062779    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:13.777489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.777458    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"]
Oct 30 21:08:13.777992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.777749    4549 cpu_manager.go:414] "containerMap: removing container" podUID="844c01d3-1113-4b80-a22d-2be606be8399" containerName="maas-api"
Oct 30 21:08:13.777992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.777759    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="844c01d3-1113-4b80-a22d-2be606be8399" containerName="maas-api"
Oct 30 21:08:13.777992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.777847    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="844c01d3-1113-4b80-a22d-2be606be8399" containerName="maas-api"
Oct 30 21:08:13.780681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.780660    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.794863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.794801    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:08:13.801496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.801470    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.801612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.801535    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.801612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.801590    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.801697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.801632    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vtj6\" (UniqueName: \"kubernetes.io/projected/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-kube-api-access-7vtj6\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.801697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.801655    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.801697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.801672    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.806692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.806670    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 21:08:13.806978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.806677    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:08:13.807057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.806761    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sd4tj\""
Oct 30 21:08:13.807981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.807964    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"]
Oct 30 21:08:13.851768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.851733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:13.903015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.902981    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903039    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903085    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903110    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903135    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vtj6\" (UniqueName: \"kubernetes.io/projected/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-kube-api-access-7vtj6\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903157    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903551    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903574    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.903641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.903629    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.905579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.905562    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.905737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.905722    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:13.922102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:13.922081    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vtj6\" (UniqueName: \"kubernetes.io/projected/d1154792-d0fb-4c0c-9b69-75ff2b0905a4-kube-api-access-7vtj6\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7\" (UID: \"d1154792-d0fb-4c0c-9b69-75ff2b0905a4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:14.098891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:14.098857    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:14.399929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:14.399855    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"]
Oct 30 21:08:14.403010 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:14.402969    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1154792_d0fb_4c0c_9b69_75ff2b0905a4.slice/crio-b0e8eb2b222ffb570dd62e24ffceb48fe7c4a0833e623b4d792b8cfad5296d6c WatchSource:0}: Error finding container b0e8eb2b222ffb570dd62e24ffceb48fe7c4a0833e623b4d792b8cfad5296d6c: Status 404 returned error can't find the container with id b0e8eb2b222ffb570dd62e24ffceb48fe7c4a0833e623b4d792b8cfad5296d6c
Oct 30 21:08:15.244910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:15.244864    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7" event={"ID":"d1154792-d0fb-4c0c-9b69-75ff2b0905a4","Type":"ContainerStarted","Data":"b0e8eb2b222ffb570dd62e24ffceb48fe7c4a0833e623b4d792b8cfad5296d6c"}
Oct 30 21:08:16.250231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:16.250196    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7" event={"ID":"d1154792-d0fb-4c0c-9b69-75ff2b0905a4","Type":"ContainerStarted","Data":"dce6853d594161fa6dc4e561fa6a21263dec7e2eee674a52351040606404c5d0"}
Oct 30 21:08:16.290067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:16.290010    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7" podStartSLOduration=2.086633971 podStartE2EDuration="3.289993223s" totalImagesPullingTime="1.203359252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:16.28824168 +0000 UTC m=+479.062641618" watchObservedRunningTime="2026-10-30 21:08:16.289993223 +0000 UTC m=+479.064393199"
Oct 30 21:08:16.956061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:16.956021    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"]
Oct 30 21:08:16.959466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:16.959442    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:16.974466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:16.974429    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 21:08:16.983127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:16.983099    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"]
Oct 30 21:08:17.032868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.032809    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.033089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.032886    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq2mj\" (UniqueName: \"kubernetes.io/projected/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-kube-api-access-bq2mj\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.033089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.032952    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.033089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.033027    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.033089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.033056    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.033089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.033085    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.042206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.042174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:17.099948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.099913    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:17.134505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134469    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.134710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134519    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.134779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134716    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.134779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134755    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.134917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134892    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.134970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134925    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq2mj\" (UniqueName: \"kubernetes.io/projected/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-kube-api-access-bq2mj\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.135023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.134992    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.135193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.135173    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.135272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.135209    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.137478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.137424    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.137478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.137462    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.154399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.154365    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq2mj\" (UniqueName: \"kubernetes.io/projected/fd4d565e-8ace-4a8a-a8fb-114a90e3f64c-kube-api-access-bq2mj\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cdl4t\" (UID: \"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.272444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.272348    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:17.569802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:17.569615    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"]
Oct 30 21:08:17.572616 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:17.572583    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4d565e_8ace_4a8a_a8fb_114a90e3f64c.slice/crio-e73590effd0178cf463e72a6bcc8d05585c8fe5c6b14b96cff7d1bdef67f291b WatchSource:0}: Error finding container e73590effd0178cf463e72a6bcc8d05585c8fe5c6b14b96cff7d1bdef67f291b: Status 404 returned error can't find the container with id e73590effd0178cf463e72a6bcc8d05585c8fe5c6b14b96cff7d1bdef67f291b
Oct 30 21:08:18.259083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:18.259037    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t" event={"ID":"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c","Type":"ContainerStarted","Data":"73dcd42232d2c3f4ffa346050c2fdf5e191a609aa349214df23e0878648b9129"}
Oct 30 21:08:18.259266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:18.259104    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t" event={"ID":"fd4d565e-8ace-4a8a-a8fb-114a90e3f64c","Type":"ContainerStarted","Data":"e73590effd0178cf463e72a6bcc8d05585c8fe5c6b14b96cff7d1bdef67f291b"}
Oct 30 21:08:18.272517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:18.272487    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:18.297077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:18.297019    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t" podStartSLOduration=2.100518631 podStartE2EDuration="2.297003527s" totalImagesPullingTime="196.484896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:18.296629869 +0000 UTC m=+481.071029811" watchObservedRunningTime="2026-10-30 21:08:18.297003527 +0000 UTC m=+481.071403465"
Oct 30 21:08:19.907944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.907905    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"]
Oct 30 21:08:19.911320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.911299    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:19.927311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.927287    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 21:08:19.940759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.940731    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"]
Oct 30 21:08:19.961615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.961583    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:19.961615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.961617    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:19.961885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.961650    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569e7b40-df79-438b-a350-7f4cf8202c9c-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:19.961885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.961867    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:19.962011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.961923    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthst\" (UniqueName: \"kubernetes.io/projected/569e7b40-df79-438b-a350-7f4cf8202c9c-kube-api-access-sthst\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:19.962011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:19.962002    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.005443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.005411    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:20.062756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.062711    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.062990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.062767    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.062990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.062889    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.062990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.062961    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569e7b40-df79-438b-a350-7f4cf8202c9c-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.063151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.063026    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.063151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.063072    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthst\" (UniqueName: \"kubernetes.io/projected/569e7b40-df79-438b-a350-7f4cf8202c9c-kube-api-access-sthst\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.063321 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.063299    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.063382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.063330    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.063422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.063372    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.065301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.065279    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569e7b40-df79-438b-a350-7f4cf8202c9c-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.065464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.065448    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569e7b40-df79-438b-a350-7f4cf8202c9c-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.081975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.081947    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthst\" (UniqueName: \"kubernetes.io/projected/569e7b40-df79-438b-a350-7f4cf8202c9c-kube-api-access-sthst\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4wgct\" (UID: \"569e7b40-df79-438b-a350-7f4cf8202c9c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.227292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.227202    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:20.523061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:20.523037    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"]
Oct 30 21:08:20.525531 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:20.525502    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569e7b40_df79_438b_a350_7f4cf8202c9c.slice/crio-794dbd29c7e38d1fb8f6a9522b1c31e6e66d82b40ef9a522fc4709a09fc2812c WatchSource:0}: Error finding container 794dbd29c7e38d1fb8f6a9522b1c31e6e66d82b40ef9a522fc4709a09fc2812c: Status 404 returned error can't find the container with id 794dbd29c7e38d1fb8f6a9522b1c31e6e66d82b40ef9a522fc4709a09fc2812c
Oct 30 21:08:21.271852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:21.271804    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:21.272513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:21.272486    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct" event={"ID":"569e7b40-df79-438b-a350-7f4cf8202c9c","Type":"ContainerStarted","Data":"66da38295cf0c06c332bcb97f972f59876b7cb2d261d877449937b460a29c216"}
Oct 30 21:08:21.272581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:21.272549    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct" event={"ID":"569e7b40-df79-438b-a350-7f4cf8202c9c","Type":"ContainerStarted","Data":"794dbd29c7e38d1fb8f6a9522b1c31e6e66d82b40ef9a522fc4709a09fc2812c"}
Oct 30 21:08:21.315328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:21.315258    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct" podStartSLOduration=2.131420201 podStartE2EDuration="2.315238489s" totalImagesPullingTime="183.818288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:21.314010726 +0000 UTC m=+484.088410665" watchObservedRunningTime="2026-10-30 21:08:21.315238489 +0000 UTC m=+484.089638430"
Oct 30 21:08:22.227849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:22.227794    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:22.476230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:22.476184    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:24.099445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:24.099400    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:24.112326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:24.112299    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:24.295469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:24.295434    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7"
Oct 30 21:08:25.049033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.049001    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-c5dtn"]
Oct 30 21:08:25.052532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.052513    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.066737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.066718    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 21:08:25.080797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.080768    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-c5dtn"]
Oct 30 21:08:25.110523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.110495    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k89xp\" (UniqueName: \"kubernetes.io/projected/0f292b66-399c-4566-8921-3f04abdaed5a-kube-api-access-k89xp\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.110874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.110543    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.110874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.110616    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-home\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.110874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.110662    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.110874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.110698    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f292b66-399c-4566-8921-3f04abdaed5a-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.110874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.110762    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.147210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.147174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:25.211622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.211592    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.211814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.211637    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-home\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.211814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.211676    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.211814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.211697    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f292b66-399c-4566-8921-3f04abdaed5a-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.211814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.211725    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.211814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.211789    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k89xp\" (UniqueName: \"kubernetes.io/projected/0f292b66-399c-4566-8921-3f04abdaed5a-kube-api-access-k89xp\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.212109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.212071    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.212170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.212146    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-home\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.212274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.212197    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.214124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.214096    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f292b66-399c-4566-8921-3f04abdaed5a-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.214306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.214287    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f292b66-399c-4566-8921-3f04abdaed5a-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.230509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.230481    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k89xp\" (UniqueName: \"kubernetes.io/projected/0f292b66-399c-4566-8921-3f04abdaed5a-kube-api-access-k89xp\") pod \"echo-simulator-kserve-64b6f8f96c-c5dtn\" (UID: \"0f292b66-399c-4566-8921-3f04abdaed5a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.366949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.366903    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:25.661312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:25.661288    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-c5dtn"]
Oct 30 21:08:25.663949 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:25.663919    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f292b66_399c_4566_8921_3f04abdaed5a.slice/crio-1693c11bd4b065c5dd857ee1b3d6f8fba5a26758503acf38ac8ac114d783957f WatchSource:0}: Error finding container 1693c11bd4b065c5dd857ee1b3d6f8fba5a26758503acf38ac8ac114d783957f: Status 404 returned error can't find the container with id 1693c11bd4b065c5dd857ee1b3d6f8fba5a26758503acf38ac8ac114d783957f
Oct 30 21:08:26.299912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:26.299876    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn" event={"ID":"0f292b66-399c-4566-8921-3f04abdaed5a","Type":"ContainerStarted","Data":"1693c11bd4b065c5dd857ee1b3d6f8fba5a26758503acf38ac8ac114d783957f"}
Oct 30 21:08:26.343552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:26.343507    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:26.581508 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:08:26.581477    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:08:27.272895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:27.272854    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:27.286723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:27.286693    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:27.315880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:27.315850    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t"
Oct 30 21:08:28.310096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:28.310061    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn" event={"ID":"0f292b66-399c-4566-8921-3f04abdaed5a","Type":"ContainerStarted","Data":"c1dbc1f7cda61c6e768ff968c7a24eb65d7340e70572b823fedac426f3adefe7"}
Oct 30 21:08:28.350970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:28.350910    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn" podStartSLOduration=1.74053958 podStartE2EDuration="3.350893754s" totalImagesPullingTime="1.610354174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:28.350070781 +0000 UTC m=+491.124470721" watchObservedRunningTime="2026-10-30 21:08:28.350893754 +0000 UTC m=+491.125293693"
Oct 30 21:08:28.367302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:28.367267    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:30.227374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.227343    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:30.240207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.240182    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:30.330242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.330214    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct"
Oct 30 21:08:30.886035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.885999    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"]
Oct 30 21:08:30.889864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.889818    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.907818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.907794    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"]
Oct 30 21:08:30.917874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.917854    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 21:08:30.960282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.960253    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxrqn\" (UniqueName: \"kubernetes.io/projected/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-kube-api-access-bxrqn\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.960417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.960310    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.960417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.960387    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.960510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.960494    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.960552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.960522    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.960601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.960559    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:30.973693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:30.973670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:31.061739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.061707    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxrqn\" (UniqueName: \"kubernetes.io/projected/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-kube-api-access-bxrqn\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.061934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.061751    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.061934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.061768    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.061934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.061811    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.061934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.061853    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.061934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.061891    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.062322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.062287    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.062435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.062328    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.062478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.062432    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.064333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.064309    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.064560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.064542    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.081922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.081896    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxrqn\" (UniqueName: \"kubernetes.io/projected/4bf73f9c-8107-47f1-9533-2aa3fa3e7606-kube-api-access-bxrqn\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4\" (UID: \"4bf73f9c-8107-47f1-9533-2aa3fa3e7606\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.217636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.217547    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:31.513471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:31.513269    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"]
Oct 30 21:08:31.516017 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:31.515983    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bf73f9c_8107_47f1_9533_2aa3fa3e7606.slice/crio-6b427ad4cc6743a1ec712ec072a046f505be60bc17c88592b475e532005e07ad WatchSource:0}: Error finding container 6b427ad4cc6743a1ec712ec072a046f505be60bc17c88592b475e532005e07ad: Status 404 returned error can't find the container with id 6b427ad4cc6743a1ec712ec072a046f505be60bc17c88592b475e532005e07ad
Oct 30 21:08:32.329376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:32.329341    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" event={"ID":"4bf73f9c-8107-47f1-9533-2aa3fa3e7606","Type":"ContainerStarted","Data":"e0337fbf911eae3a160e5b9833e5c4bc4731edb7498f1491bd13673006b174c4"}
Oct 30 21:08:32.329376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:32.329382    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" event={"ID":"4bf73f9c-8107-47f1-9533-2aa3fa3e7606","Type":"ContainerStarted","Data":"6b427ad4cc6743a1ec712ec072a046f505be60bc17c88592b475e532005e07ad"}
Oct 30 21:08:32.369879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:32.369807    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podStartSLOduration=2.189572808 podStartE2EDuration="2.369792137s" totalImagesPullingTime="180.219329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:32.368952656 +0000 UTC m=+495.143352594" watchObservedRunningTime="2026-10-30 21:08:32.369792137 +0000 UTC m=+495.144192118"
Oct 30 21:08:33.217843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:33.217801    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:35.367742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:35.367702    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:35.381763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:35.381734    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:36.356079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:36.356052    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn"
Oct 30 21:08:41.218672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:41.218634    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:41.231520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:41.231494    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:41.374767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:41.374737    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:08:46.538677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:46.538641    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:46.769406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:46.769377    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:47.362652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.362611    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"]
Oct 30 21:08:47.367117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.367095    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.383790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.383772    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-w59m6\""
Oct 30 21:08:47.394621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.394595    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"]
Oct 30 21:08:47.513908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.513876    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.513916    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.513943    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.514049    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2x62\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.514076    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.514111    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.514227    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.514308    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.514360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.514341    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615073    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615112    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615139    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615162    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2x62\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615179    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615200    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615240    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615262    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615287    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615591    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615638    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615689    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.615723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.615712    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.616242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.616136    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.617669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.617651    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.617969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.617948    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.634686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.634659    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.634976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.634958    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2x62\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62\") pod \"maas-default-gateway-openshift-default-57f78bcd55-2ff29\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.680782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.680759    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:47.988258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:47.988086    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"]
Oct 30 21:08:47.990626 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:47.990597    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123cd94c_22ac_4613_92c7_be45db78256c.slice/crio-bc23e77ab49ce2fe69534f5842b4ca992586fcaf429189e87f88298e45223886 WatchSource:0}: Error finding container bc23e77ab49ce2fe69534f5842b4ca992586fcaf429189e87f88298e45223886: Status 404 returned error can't find the container with id bc23e77ab49ce2fe69534f5842b4ca992586fcaf429189e87f88298e45223886
Oct 30 21:08:48.395387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:48.395349    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" event={"ID":"123cd94c-22ac-4613-92c7-be45db78256c","Type":"ContainerStarted","Data":"bc23e77ab49ce2fe69534f5842b4ca992586fcaf429189e87f88298e45223886"}
Oct 30 21:08:50.668646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:50.668609    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:08:50.668994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:50.668681    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:08:50.668994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:50.668711    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:08:51.412878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:51.412840    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" event={"ID":"123cd94c-22ac-4613-92c7-be45db78256c","Type":"ContainerStarted","Data":"9916b0cffac09d10598f6e215b7ddbae00b2247008529d869b69a1cd621ba377"}
Oct 30 21:08:51.488103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:51.488035    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podStartSLOduration=1.812802821 podStartE2EDuration="4.488014166s" totalImagesPullingTime="2.675211345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:51.469276843 +0000 UTC m=+514.243676784" watchObservedRunningTime="2026-10-30 21:08:51.488014166 +0000 UTC m=+514.262414109"
Oct 30 21:08:51.688918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:51.685894    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:08:51.688918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:51.686070    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:51.688918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:51.686116    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:52.268219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.268191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:52.401575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.401536    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:52.408763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.408728    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"]
Oct 30 21:08:52.412578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.412557    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.430147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.430113    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"]
Oct 30 21:08:52.439468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.439428    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 21:08:52.439611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.439494    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-v7pb6\""
Oct 30 21:08:52.504975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.504941    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-kjg7p"]
Oct 30 21:08:52.508412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.508388    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.526435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.526352    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-kjg7p"]
Oct 30 21:08:52.558905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.558867    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzqhx\" (UniqueName: \"kubernetes.io/projected/992ca47a-dcf6-4284-a77f-34af9342e718-kube-api-access-qzqhx\") pod \"payload-pre-processing-7b98bb678c-tbwcl\" (UID: \"992ca47a-dcf6-4284-a77f-34af9342e718\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.559118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.558987    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/992ca47a-dcf6-4284-a77f-34af9342e718-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-tbwcl\" (UID: \"992ca47a-dcf6-4284-a77f-34af9342e718\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.660432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.660394    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzqhx\" (UniqueName: \"kubernetes.io/projected/992ca47a-dcf6-4284-a77f-34af9342e718-kube-api-access-qzqhx\") pod \"payload-pre-processing-7b98bb678c-tbwcl\" (UID: \"992ca47a-dcf6-4284-a77f-34af9342e718\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.660644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.660442    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8254b5c-1d56-4b59-8760-0d16e3761711-plugins-config-volume\") pod \"payload-processing-5f78687d5b-kjg7p\" (UID: \"a8254b5c-1d56-4b59-8760-0d16e3761711\") " pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.660644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.660470    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/992ca47a-dcf6-4284-a77f-34af9342e718-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-tbwcl\" (UID: \"992ca47a-dcf6-4284-a77f-34af9342e718\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.660644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.660602    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4jw2\" (UniqueName: \"kubernetes.io/projected/a8254b5c-1d56-4b59-8760-0d16e3761711-kube-api-access-d4jw2\") pod \"payload-processing-5f78687d5b-kjg7p\" (UID: \"a8254b5c-1d56-4b59-8760-0d16e3761711\") " pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.661307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.661287    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/992ca47a-dcf6-4284-a77f-34af9342e718-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-tbwcl\" (UID: \"992ca47a-dcf6-4284-a77f-34af9342e718\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.683988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.683948    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:52.684284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.684253    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:52.695081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.695045    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzqhx\" (UniqueName: \"kubernetes.io/projected/992ca47a-dcf6-4284-a77f-34af9342e718-kube-api-access-qzqhx\") pod \"payload-pre-processing-7b98bb678c-tbwcl\" (UID: \"992ca47a-dcf6-4284-a77f-34af9342e718\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.726333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.726299    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:52.761246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.761212    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4jw2\" (UniqueName: \"kubernetes.io/projected/a8254b5c-1d56-4b59-8760-0d16e3761711-kube-api-access-d4jw2\") pod \"payload-processing-5f78687d5b-kjg7p\" (UID: \"a8254b5c-1d56-4b59-8760-0d16e3761711\") " pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.761448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.761328    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8254b5c-1d56-4b59-8760-0d16e3761711-plugins-config-volume\") pod \"payload-processing-5f78687d5b-kjg7p\" (UID: \"a8254b5c-1d56-4b59-8760-0d16e3761711\") " pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.762326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.762302    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8254b5c-1d56-4b59-8760-0d16e3761711-plugins-config-volume\") pod \"payload-processing-5f78687d5b-kjg7p\" (UID: \"a8254b5c-1d56-4b59-8760-0d16e3761711\") " pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.799949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.799874    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4jw2\" (UniqueName: \"kubernetes.io/projected/a8254b5c-1d56-4b59-8760-0d16e3761711-kube-api-access-d4jw2\") pod \"payload-processing-5f78687d5b-kjg7p\" (UID: \"a8254b5c-1d56-4b59-8760-0d16e3761711\") " pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:52.822241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:52.822201    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:53.045849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:53.045804    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"]
Oct 30 21:08:53.048140 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:53.048107    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992ca47a_dcf6_4284_a77f_34af9342e718.slice/crio-5b5afb25ea0c7d58b1c8ebbea7e12f5ea1af4f8a7ea40500ffce6d03b22e85de WatchSource:0}: Error finding container 5b5afb25ea0c7d58b1c8ebbea7e12f5ea1af4f8a7ea40500ffce6d03b22e85de: Status 404 returned error can't find the container with id 5b5afb25ea0c7d58b1c8ebbea7e12f5ea1af4f8a7ea40500ffce6d03b22e85de
Oct 30 21:08:53.149152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:53.149129    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-kjg7p"]
Oct 30 21:08:53.151481 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:08:53.151450    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8254b5c_1d56_4b59_8760_0d16e3761711.slice/crio-120a0696eec9daffd4d605b894fba1ba8441241ab1614dfa0a7c6b722d817ee2 WatchSource:0}: Error finding container 120a0696eec9daffd4d605b894fba1ba8441241ab1614dfa0a7c6b722d817ee2: Status 404 returned error can't find the container with id 120a0696eec9daffd4d605b894fba1ba8441241ab1614dfa0a7c6b722d817ee2
Oct 30 21:08:53.423014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:53.422974    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl" event={"ID":"992ca47a-dcf6-4284-a77f-34af9342e718","Type":"ContainerStarted","Data":"5b5afb25ea0c7d58b1c8ebbea7e12f5ea1af4f8a7ea40500ffce6d03b22e85de"}
Oct 30 21:08:53.424392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:53.424362    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p" event={"ID":"a8254b5c-1d56-4b59-8760-0d16e3761711","Type":"ContainerStarted","Data":"120a0696eec9daffd4d605b894fba1ba8441241ab1614dfa0a7c6b722d817ee2"}
Oct 30 21:08:53.687913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:53.684410    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:53.687913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:53.684476    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:54.694219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:54.694154    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:54.694717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:54.694220    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:55.692367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:55.692324    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:55.692699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:55.692411    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:56.601520 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:08:56.601484    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:08:56.690919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:56.690281    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:56.690919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:56.690349    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:57.446192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.446155    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl" event={"ID":"992ca47a-dcf6-4284-a77f-34af9342e718","Type":"ContainerStarted","Data":"11881700c03d28ced623faeee23d40ef72360b2bc04ab18a77ee8455205fea58"}
Oct 30 21:08:57.446398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.446364    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:08:57.447805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.447771    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p" event={"ID":"a8254b5c-1d56-4b59-8760-0d16e3761711","Type":"ContainerStarted","Data":"2d531bdfa806aa2d97355fe9b240da05513cd195116bcead24b94339a2a527ae"}
Oct 30 21:08:57.447946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.447882    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:08:57.484696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.484644    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl" podStartSLOduration=2.116890311 podStartE2EDuration="5.484630154s" totalImagesPullingTime="3.367739843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:57.483021391 +0000 UTC m=+520.257421330" watchObservedRunningTime="2026-10-30 21:08:57.484630154 +0000 UTC m=+520.259030093"
Oct 30 21:08:57.523761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.523705    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p" podStartSLOduration=2.225147629 podStartE2EDuration="5.523690683s" totalImagesPullingTime="3.298543054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:57.522890449 +0000 UTC m=+520.297290388" watchObservedRunningTime="2026-10-30 21:08:57.523690683 +0000 UTC m=+520.298090623"
Oct 30 21:08:57.682086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.682049    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 21:08:57.682475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:57.682102    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 21:08:59.682154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:59.682101    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:08:59.682660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:08:59.682180    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:09:00.724019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:00.723980    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:09:00.724528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:00.724062    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:09:01.083609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:01.083577    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:09:01.083905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:01.083881    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:09:01.084717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:01.084696    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:09:02.398453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.398421    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"]
Oct 30 21:09:02.406955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.406924    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.426707 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.426672    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"]
Oct 30 21:09:02.480688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480647    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.480903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480712    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.480903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480747    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.480903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480788    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.480903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480816    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.481070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480923    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pwv5\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.481070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480957    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.481070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.480980    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.481223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.481092    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582248    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582332    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582450    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582532    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582568    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582643    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pwv5\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582687    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582724    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582727    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582873    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.582947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.582886    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.583127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.583099    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.583255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.583121    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.583305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.583264    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.585478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.585454    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.585675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.585659    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.602030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.601997    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.602323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.602304    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pwv5\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4nxfb\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:02.720907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:02.720807    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:03.030667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.030576    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"]
Oct 30 21:09:03.032943 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:03.032915    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf385494_acff_455b_b1fe_0f561f909fc9.slice/crio-4a8aa02c9c2f99c9cf6325b4e4b2dd0a7bc4eba346250e7021f1a49f50dedc09 WatchSource:0}: Error finding container 4a8aa02c9c2f99c9cf6325b4e4b2dd0a7bc4eba346250e7021f1a49f50dedc09: Status 404 returned error can't find the container with id 4a8aa02c9c2f99c9cf6325b4e4b2dd0a7bc4eba346250e7021f1a49f50dedc09
Oct 30 21:09:03.036266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.036231    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:03.036340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.036305    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:03.036377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.036337    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:03.480193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.480147    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" event={"ID":"af385494-acff-455b-b1fe-0f561f909fc9","Type":"ContainerStarted","Data":"dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec"}
Oct 30 21:09:03.480748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.480226    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" event={"ID":"af385494-acff-455b-b1fe-0f561f909fc9","Type":"ContainerStarted","Data":"4a8aa02c9c2f99c9cf6325b4e4b2dd0a7bc4eba346250e7021f1a49f50dedc09"}
Oct 30 21:09:03.721044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:03.721011    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:04.193857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:04.193662    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:04.723018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:04.722987    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:09:04.723393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:04.723038    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:09:05.721794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:05.721750    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:09:05.722025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:05.721805    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:09:06.721617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:06.721578    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:09:06.722055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:06.721631    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:09:07.721134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:07.721093    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:09:07.721360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:07.721149    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:09:08.455291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:08.455261    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-kjg7p"
Oct 30 21:09:08.455751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:08.455389    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl"
Oct 30 21:09:08.494916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:08.494867    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podStartSLOduration=6.494851672 podStartE2EDuration="6.494851672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:03.55055009 +0000 UTC m=+526.324950029" watchObservedRunningTime="2026-10-30 21:09:08.494851672 +0000 UTC m=+531.269251589"
Oct 30 21:09:08.722086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:08.722000    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:09:08.722086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:08.722053    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:09:09.722146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:09.722108    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:09:09.722531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:09.722165    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 21:09:11.254285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:11.254253    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:11.254760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:11.254665    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:11.255518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:11.255497    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:09:12.440713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:12.440682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:12.565125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:12.565082    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:18.827233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:18.827199    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:19.342435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:19.342399    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:20.955640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:20.955612    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:21.089475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:21.089440    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:22.478255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:22.478210    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:22.611993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:22.611961    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:26.571808 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:09:26.571774    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:09:44.239052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.239017    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.363010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.362966    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.748640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.748602    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.900055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.900025    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.929089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.929031    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"]
Oct 30 21:09:44.932088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.932068    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:44.947342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.947317    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-56ac74-openshift-default-dockercfg-gsffb\""
Oct 30 21:09:44.960747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:44.960705    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"]
Oct 30 21:09:45.003404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003307    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003392    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003431    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003506    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003546    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003571    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dld\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003629    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003671    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.003859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.003766    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.031336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.031296    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.058348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.058311    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"]
Oct 30 21:09:45.061508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.061477    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.085567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.085528    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-839163-openshift-default-dockercfg-lpmcf\""
Oct 30 21:09:45.089213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.089181    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"]
Oct 30 21:09:45.104757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.104723    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsznr\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.104948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.104783    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.104948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.104885    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.105046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.104979    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.105085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105050    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.105121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105084    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.105121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105114    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.105218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105144    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.105218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105171    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.105218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105197    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.105566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105542    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.105903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.105875    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.106185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106162    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.106265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106197    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.106338 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106302    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dld\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.106407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106355    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.106407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106399    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.106515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106438    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.106515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106473    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.106515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106507    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.106673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106543    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.106936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.106727    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.109261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.109236    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.109492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.109473    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.111458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.111420    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.128279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.128249    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.128431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.128306    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dld\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.183409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.183353    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.207590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207544    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.207864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207595    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsznr\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.207864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207649    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.207864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207740    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.207864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207773    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.207864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207800    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.207864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207857    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207892    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.207932    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.208088    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.208142    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.208288    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.208419    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.208970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.208759    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.211145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.211117    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.211567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.211426    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.229023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.228989    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.229160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.229144    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsznr\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.246991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.246959    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:45.320552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.320523    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.345264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.345002    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"]
Oct 30 21:09:45.352806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.352748    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.378757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.377715    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:45.385714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.385505    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-2cda44-openshift-default-dockercfg-4kbl9\""
Oct 30 21:09:45.385714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.385546    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"]
Oct 30 21:09:45.413367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413211    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413410    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413445    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413515    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413646    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413675    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btndk\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413908    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.413988    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.414589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.414033    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.445212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.445172    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.516477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.516387    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517427    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517545    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517661    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517725    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517815    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517872    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btndk\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.517941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517904    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.518899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.517970    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.518899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.518032    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.518899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.518116    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.518899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.518423    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.518899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.518798    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.519474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.519434    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.521219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.521196    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.521909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.521886    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.537631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.537601    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btndk\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.537947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.537924    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.566816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.566793    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"]
Oct 30 21:09:45.570071 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:45.570044    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e1ff6d_edaf_46f9_9653_4dc6bb3eab83.slice/crio-ae298260cb91cbccb5793719a4cd1cb0d5a8feb4b9a68fda482098d34b99ca45 WatchSource:0}: Error finding container ae298260cb91cbccb5793719a4cd1cb0d5a8feb4b9a68fda482098d34b99ca45: Status 404 returned error can't find the container with id ae298260cb91cbccb5793719a4cd1cb0d5a8feb4b9a68fda482098d34b99ca45
Oct 30 21:09:45.572481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.572442    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:45.572578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.572535    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:45.572630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.572577    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:45.573140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.573120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.675559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.675524    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" event={"ID":"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83","Type":"ContainerStarted","Data":"0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"}
Oct 30 21:09:45.675735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.675584    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" event={"ID":"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83","Type":"ContainerStarted","Data":"ae298260cb91cbccb5793719a4cd1cb0d5a8feb4b9a68fda482098d34b99ca45"}
Oct 30 21:09:45.685313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.685282    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"]
Oct 30 21:09:45.689229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.687774    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:45.691881 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:45.691813    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c4c4f61_eceb_42c4_bdd0_4e57a010ee99.slice/crio-0bae33ae032f16ad0609ab2ce290e616ae86cbd27f76edb7e44f9c974a9789ae WatchSource:0}: Error finding container 0bae33ae032f16ad0609ab2ce290e616ae86cbd27f76edb7e44f9c974a9789ae: Status 404 returned error can't find the container with id 0bae33ae032f16ad0609ab2ce290e616ae86cbd27f76edb7e44f9c974a9789ae
Oct 30 21:09:45.696171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.696136    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:45.696281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.696220    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:45.696281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.696262    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:45.698181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.698144    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.723397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.723319    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podStartSLOduration=1.723299446 podStartE2EDuration="1.723299446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:45.722564595 +0000 UTC m=+568.496964544" watchObservedRunningTime="2026-10-30 21:09:45.723299446 +0000 UTC m=+568.497699386"
Oct 30 21:09:45.815143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:45.815105    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:46.000250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.000222    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"]
Oct 30 21:09:46.002921 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:46.002885    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da7f759_7cf9_49d9_b745_b2ac768234a9.slice/crio-be8481ed01dcb6a2a3eab2a5b6110d792e76e8fb97c248c39cf97077ce361a43 WatchSource:0}: Error finding container be8481ed01dcb6a2a3eab2a5b6110d792e76e8fb97c248c39cf97077ce361a43: Status 404 returned error can't find the container with id be8481ed01dcb6a2a3eab2a5b6110d792e76e8fb97c248c39cf97077ce361a43
Oct 30 21:09:46.005765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.005736    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:46.005865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.005800    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:46.005924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.005909    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:09:46.247752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.247713    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:46.253302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.253272    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:46.681367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.681324    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" event={"ID":"0da7f759-7cf9-49d9-b745-b2ac768234a9","Type":"ContainerStarted","Data":"ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"}
Oct 30 21:09:46.681586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.681423    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" event={"ID":"0da7f759-7cf9-49d9-b745-b2ac768234a9","Type":"ContainerStarted","Data":"be8481ed01dcb6a2a3eab2a5b6110d792e76e8fb97c248c39cf97077ce361a43"}
Oct 30 21:09:46.683047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.683011    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" event={"ID":"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99","Type":"ContainerStarted","Data":"a5e666c7a17afe76d3410711d0c7ce0dae7ad2cdf74e7273e3255702cbcd6db9"}
Oct 30 21:09:46.683135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.683070    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" event={"ID":"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99","Type":"ContainerStarted","Data":"0bae33ae032f16ad0609ab2ce290e616ae86cbd27f76edb7e44f9c974a9789ae"}
Oct 30 21:09:46.683194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.683180    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:46.684120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.684103    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:09:46.689224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.689209    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:46.721807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.721745    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podStartSLOduration=1.721733908 podStartE2EDuration="1.721733908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:46.721204331 +0000 UTC m=+569.495604454" watchObservedRunningTime="2026-10-30 21:09:46.721733908 +0000 UTC m=+569.496133847"
Oct 30 21:09:46.802157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:46.802105    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podStartSLOduration=1.8020910030000001 podStartE2EDuration="1.802091003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:46.800469409 +0000 UTC m=+569.574869407" watchObservedRunningTime="2026-10-30 21:09:46.802091003 +0000 UTC m=+569.576490941"
Oct 30 21:09:47.386231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:47.386131    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:47.391498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:47.391476    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:47.689859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:47.688271    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:47.690770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:47.690742    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:09:47.693735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:47.693709    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:48.240692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:48.240642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:48.372170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:48.372136    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:48.692935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:48.692902    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:48.693895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:48.693875    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:09:49.006730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.006641    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.151080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.151049    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.304934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.304840    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.436653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.436619    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.593456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.593423    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.726179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.726145    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.872438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:49.872361    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:50.012705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.012655    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:50.486614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.486581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:50.622270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.622235    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:50.676584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.676545    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"]
Oct 30 21:09:50.679259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.679238    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.705751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.705720    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-56ac74-dockercfg-dbhqb\""
Oct 30 21:09:50.705984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.705777    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-56ac74\""
Oct 30 21:09:50.707738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.707713    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"]
Oct 30 21:09:50.790906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.790700    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn5nd\" (UniqueName: \"kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd\") pod \"payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.792360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.791551    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.892582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.892545    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.892783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.892624    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn5nd\" (UniqueName: \"kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd\") pod \"payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.893296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.893276    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.912045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.912016    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn5nd\" (UniqueName: \"kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd\") pod \"payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:50.993133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:50.993093    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:51.287934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:51.287907    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"]
Oct 30 21:09:51.290273 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:51.290245    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a448953_34f7_4a32_985c_242827d5d848.slice/crio-e3bd7e7d2ddbeb2fde3be5616ba2c528dc9db5c1cc3fc4d495440774ef2c1ccf WatchSource:0}: Error finding container e3bd7e7d2ddbeb2fde3be5616ba2c528dc9db5c1cc3fc4d495440774ef2c1ccf: Status 404 returned error can't find the container with id e3bd7e7d2ddbeb2fde3be5616ba2c528dc9db5c1cc3fc4d495440774ef2c1ccf
Oct 30 21:09:51.710639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:51.710599    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs" event={"ID":"6a448953-34f7-4a32-985c-242827d5d848","Type":"ContainerStarted","Data":"50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb"}
Oct 30 21:09:51.710820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:51.710658    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs" event={"ID":"6a448953-34f7-4a32-985c-242827d5d848","Type":"ContainerStarted","Data":"e3bd7e7d2ddbeb2fde3be5616ba2c528dc9db5c1cc3fc4d495440774ef2c1ccf"}
Oct 30 21:09:51.710820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:51.710721    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:09:51.751204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:51.751146    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs" podStartSLOduration=1.751131778 podStartE2EDuration="1.751131778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:51.750077953 +0000 UTC m=+574.524477893" watchObservedRunningTime="2026-10-30 21:09:51.751131778 +0000 UTC m=+574.525531717"
Oct 30 21:09:52.012186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.012095    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.063369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.063330    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"]
Oct 30 21:09:52.065953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.065934    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.085027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.084995    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"]
Oct 30 21:09:52.085336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.085317    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-79477e\""
Oct 30 21:09:52.088472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.088441    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.088880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.088852    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"]
Oct 30 21:09:52.107316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.107286    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-79477e-dockercfg-jzcc5\""
Oct 30 21:09:52.110307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.110281    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"]
Oct 30 21:09:52.136566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.136523    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.209718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.209672    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.209718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.209771    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.209718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.209820    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzs2h\" (UniqueName: \"kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h\") pod \"payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.210174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.209958    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktl9w\" (UniqueName: \"kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w\") pod \"payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.311300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.311199    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.311300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.311257    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzs2h\" (UniqueName: \"kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h\") pod \"payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.311534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.311336    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktl9w\" (UniqueName: \"kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w\") pod \"payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.311534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.311364    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.312006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.311984    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.312078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.312061    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.330293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.330260    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktl9w\" (UniqueName: \"kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w\") pod \"payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.330446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.330394    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzs2h\" (UniqueName: \"kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h\") pod \"payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.378558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.378523    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:52.403965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.403935    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:52.601335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.601297    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"]
Oct 30 21:09:52.604408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.604384    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.623739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.622942    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-56ac74-serving-cert\""
Oct 30 21:09:52.625541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.625515    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"]
Oct 30 21:09:52.688026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.687994    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"]
Oct 30 21:09:52.689506 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:52.689473    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51eabe42_4d52_4337_ab89_418bc86f0849.slice/crio-ccf95aa66790817fded629620b11f5badec63daf92a620fbdb387caabccaccf9 WatchSource:0}: Error finding container ccf95aa66790817fded629620b11f5badec63daf92a620fbdb387caabccaccf9: Status 404 returned error can't find the container with id ccf95aa66790817fded629620b11f5badec63daf92a620fbdb387caabccaccf9
Oct 30 21:09:52.712634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.712592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.715345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.715313    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhz7f\" (UniqueName: \"kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.715345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.715329    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"]
Oct 30 21:09:52.715495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.715406    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.715495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.715444    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.715726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.715593    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.718186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.718149    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" event={"ID":"51eabe42-4d52-4337-ab89-418bc86f0849","Type":"ContainerStarted","Data":"ccf95aa66790817fded629620b11f5badec63daf92a620fbdb387caabccaccf9"}
Oct 30 21:09:52.720899 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:52.720874    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9bd599_6ee1_4572_ae7c_f36f7d0d4b64.slice/crio-22db0dbd6b6daf65468497dcccec62b8ef8cc48ec87cae85eb386807d89f0ff3 WatchSource:0}: Error finding container 22db0dbd6b6daf65468497dcccec62b8ef8cc48ec87cae85eb386807d89f0ff3: Status 404 returned error can't find the container with id 22db0dbd6b6daf65468497dcccec62b8ef8cc48ec87cae85eb386807d89f0ff3
Oct 30 21:09:52.817115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.817073    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.817289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.817173    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhz7f\" (UniqueName: \"kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.817289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.817214    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.817289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.817243    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.818156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.818093    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.819881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.819849    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.820023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.820004    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.836182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.836147    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhz7f\" (UniqueName: \"kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f\") pod \"maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.836490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.836468    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.916538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.916450    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:52.958812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.958766    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"]
Oct 30 21:09:52.962378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.962355    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:52.966494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.966467    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.992615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:52.992580    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"]
Oct 30 21:09:53.038172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.037970    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-58fea9\""
Oct 30 21:09:53.038172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.038138    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-58fea9-dockercfg-qhpf2\""
Oct 30 21:09:53.058243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.058153    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2jxm\" (UniqueName: \"kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm\") pod \"payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.058616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.058589    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.160219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.160183    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2jxm\" (UniqueName: \"kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm\") pod \"payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.160388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.160280    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.161028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.161011    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.180272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.180205    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2jxm\" (UniqueName: \"kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm\") pod \"payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.220668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.220629    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"]
Oct 30 21:09:53.223458 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:53.223418    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7ca725_d313_455a_89dc_9da4580a8755.slice/crio-31d682746d3a4e884c54e6b3b20b5bc21d4bad67a83d4774f9d4b8de894ebeb5 WatchSource:0}: Error finding container 31d682746d3a4e884c54e6b3b20b5bc21d4bad67a83d4774f9d4b8de894ebeb5: Status 404 returned error can't find the container with id 31d682746d3a4e884c54e6b3b20b5bc21d4bad67a83d4774f9d4b8de894ebeb5
Oct 30 21:09:53.276665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.276631    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.576113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.576080    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"]
Oct 30 21:09:53.578299 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:53.578270    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36205f5_889f_4f2d_b0c7_8940b127181f.slice/crio-558c7668bf56a5e7320f16c31e4594f4bebcd1792788d6ac80cb93faedfaee8d WatchSource:0}: Error finding container 558c7668bf56a5e7320f16c31e4594f4bebcd1792788d6ac80cb93faedfaee8d: Status 404 returned error can't find the container with id 558c7668bf56a5e7320f16c31e4594f4bebcd1792788d6ac80cb93faedfaee8d
Oct 30 21:09:53.724574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.724540    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" event={"ID":"51eabe42-4d52-4337-ab89-418bc86f0849","Type":"ContainerStarted","Data":"9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58"}
Oct 30 21:09:53.724779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.724689    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:09:53.726031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.726009    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp" event={"ID":"eb7ca725-d313-455a-89dc-9da4580a8755","Type":"ContainerStarted","Data":"f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f"}
Oct 30 21:09:53.726161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.726045    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp" event={"ID":"eb7ca725-d313-455a-89dc-9da4580a8755","Type":"ContainerStarted","Data":"31d682746d3a4e884c54e6b3b20b5bc21d4bad67a83d4774f9d4b8de894ebeb5"}
Oct 30 21:09:53.726161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.726103    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:53.727593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.727568    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" event={"ID":"b36205f5-889f-4f2d-b0c7-8940b127181f","Type":"ContainerStarted","Data":"3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25"}
Oct 30 21:09:53.727693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.727606    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" event={"ID":"b36205f5-889f-4f2d-b0c7-8940b127181f","Type":"ContainerStarted","Data":"558c7668bf56a5e7320f16c31e4594f4bebcd1792788d6ac80cb93faedfaee8d"}
Oct 30 21:09:53.727693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.727688    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:09:53.729077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.729057    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" event={"ID":"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64","Type":"ContainerStarted","Data":"67e77be3824fe68c6a7210e6fa02a9c180ec62c3ad9b906e41e1180d724f7f2a"}
Oct 30 21:09:53.729180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.729100    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" event={"ID":"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64","Type":"ContainerStarted","Data":"22db0dbd6b6daf65468497dcccec62b8ef8cc48ec87cae85eb386807d89f0ff3"}
Oct 30 21:09:53.729180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.729129    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:09:53.768282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.768232    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" podStartSLOduration=1.7682191569999999 podStartE2EDuration="1.768219157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:53.765796118 +0000 UTC m=+576.540196056" watchObservedRunningTime="2026-10-30 21:09:53.768219157 +0000 UTC m=+576.542619097"
Oct 30 21:09:53.806927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.806875    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" podStartSLOduration=1.806859674 podStartE2EDuration="1.806859674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:53.805854713 +0000 UTC m=+576.580254645" watchObservedRunningTime="2026-10-30 21:09:53.806859674 +0000 UTC m=+576.581259607"
Oct 30 21:09:53.851263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.851162    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp" podStartSLOduration=1.4805617770000001 podStartE2EDuration="1.851150138s" totalImagesPullingTime="370.588361ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:53.84935685 +0000 UTC m=+576.623756802" watchObservedRunningTime="2026-10-30 21:09:53.851150138 +0000 UTC m=+576.625550067"
Oct 30 21:09:53.890662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:53.890610    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" podStartSLOduration=1.890596661 podStartE2EDuration="1.890596661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:53.888554387 +0000 UTC m=+576.662954326" watchObservedRunningTime="2026-10-30 21:09:53.890596661 +0000 UTC m=+576.664996601"
Oct 30 21:09:54.168005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.167921    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"]
Oct 30 21:09:54.170625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.170598    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.189706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.189676    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-839163-serving-cert\""
Oct 30 21:09:54.196588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.196553    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"]
Oct 30 21:09:54.276182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.276152    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.276369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.276195    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.276369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.276298    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgjdc\" (UniqueName: \"kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.276369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.276326    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.287735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.287699    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:54.377249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.377211    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.377440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.377260    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgjdc\" (UniqueName: \"kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.377440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.377289    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.377440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.377393    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.378038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.378013    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.380081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.380050    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.380194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.380085    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.398731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.398702    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgjdc\" (UniqueName: \"kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc\") pod \"maas-api-e2e-worker-w4-839163-674d48c657-vlmkj\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.423119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.423034    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:54.495407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.492042    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:54.563924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.559450    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:54.796532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.796452    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"]
Oct 30 21:09:54.798946 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:54.798914    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f23302_342c_41a8_b18c_68ec89497e8b.slice/crio-f3f8ce9becd9f724a74ec90532357d6c9f58e7d6d96752ddfa6bc3eb956e7dd9 WatchSource:0}: Error finding container f3f8ce9becd9f724a74ec90532357d6c9f58e7d6d96752ddfa6bc3eb956e7dd9: Status 404 returned error can't find the container with id f3f8ce9becd9f724a74ec90532357d6c9f58e7d6d96752ddfa6bc3eb956e7dd9
Oct 30 21:09:54.937458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:54.937424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:55.077464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.077434    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:55.707300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.707271    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:55.741295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.741253    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj" event={"ID":"18f23302-342c-41a8-b18c-68ec89497e8b","Type":"ContainerStarted","Data":"f80c13cb1c7b3ca0f4ae2ddac521e0babefc6bb0439da76bf52eaad64cfa4ac0"}
Oct 30 21:09:55.741447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.741325    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:09:55.741447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.741346    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj" event={"ID":"18f23302-342c-41a8-b18c-68ec89497e8b","Type":"ContainerStarted","Data":"f3f8ce9becd9f724a74ec90532357d6c9f58e7d6d96752ddfa6bc3eb956e7dd9"}
Oct 30 21:09:55.775616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.775586    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"]
Oct 30 21:09:55.778298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.778280    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.792194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.792152    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.792481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.792455    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.792569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.792505    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk2mg\" (UniqueName: \"kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.792650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.792632    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.793410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.793367    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj" podStartSLOduration=1.4382611650000001 podStartE2EDuration="1.793355328s" totalImagesPullingTime="355.094163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:55.792008151 +0000 UTC m=+578.566408086" watchObservedRunningTime="2026-10-30 21:09:55.793355328 +0000 UTC m=+578.567755303"
Oct 30 21:09:55.803670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.803615    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-79477e-serving-cert\""
Oct 30 21:09:55.806668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.806639    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"]
Oct 30 21:09:55.857549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.857498    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:55.893662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.893631    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.893662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.893663    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk2mg\" (UniqueName: \"kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.893902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.893710    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.893902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.893769    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.894620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.894597    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.896396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.896375    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.896498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.896449    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.912953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.912924    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk2mg\" (UniqueName: \"kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg\") pod \"maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:55.990081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:55.989987    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:56.090443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:56.090411    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:56.389527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:56.389498    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"]
Oct 30 21:09:56.391705 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:09:56.391671    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96cd0105_e983_441f_80c2_55572a827e1a.slice/crio-65ef2c474628bfabaddc289687a2c247791d716ab66d6764202eb1e74c961ab2 WatchSource:0}: Error finding container 65ef2c474628bfabaddc289687a2c247791d716ab66d6764202eb1e74c961ab2: Status 404 returned error can't find the container with id 65ef2c474628bfabaddc289687a2c247791d716ab66d6764202eb1e74c961ab2
Oct 30 21:09:56.574572 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:09:56.574532    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:09:56.748130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:56.748096    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn" event={"ID":"96cd0105-e983-441f-80c2-55572a827e1a","Type":"ContainerStarted","Data":"65ef2c474628bfabaddc289687a2c247791d716ab66d6764202eb1e74c961ab2"}
Oct 30 21:09:57.220704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.220659    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.352532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.352492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.406491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.406460    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"]
Oct 30 21:09:57.409332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.409309    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.441145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.441111    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"]
Oct 30 21:09:57.502969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.502874    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-58fea9-serving-cert\""
Oct 30 21:09:57.512769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.512737    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6l9c\" (UniqueName: \"kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.512988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.512785    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.512988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.512811    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.512988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.512959    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.513604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.513586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.616200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.616165    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6l9c\" (UniqueName: \"kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.616418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.616239    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.616418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.616275    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.616418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.616330    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.617112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.617088    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.617813 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:09:57.617791    4549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-serving-cert: secret "maas-api-e2e-worker-w0-58fea9-serving-cert" not found
Oct 30 21:09:57.617988 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:09:57.617884    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls podName:abe73d82-7ffe-44bb-a9fe-ece88010a356 nodeName:}" failed. No retries permitted until 2026-10-30 21:09:58.117865354 +0000 UTC m=+580.892265271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls") pod "maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" (UID: "abe73d82-7ffe-44bb-a9fe-ece88010a356") : secret "maas-api-e2e-worker-w0-58fea9-serving-cert" not found
Oct 30 21:09:57.630902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.630868    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.636150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.636009    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.641852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.640053    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6l9c\" (UniqueName: \"kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:57.755122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.755025    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn" event={"ID":"96cd0105-e983-441f-80c2-55572a827e1a","Type":"ContainerStarted","Data":"f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc"}
Oct 30 21:09:57.755122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.755095    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:09:57.795461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.795416    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.798614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:57.798566    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn" podStartSLOduration=2.490877967 podStartE2EDuration="2.798551309s" totalImagesPullingTime="307.673342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:57.798085782 +0000 UTC m=+580.572485721" watchObservedRunningTime="2026-10-30 21:09:57.798551309 +0000 UTC m=+580.572951249"
Oct 30 21:09:58.108855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.108438    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:58.128624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.128569    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:58.131651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.131614    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\") pod \"maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:58.241805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.241767    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:58.328677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.327906    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:58.725057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.724431    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"]
Oct 30 21:09:58.761000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:58.760968    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" event={"ID":"abe73d82-7ffe-44bb-a9fe-ece88010a356","Type":"ContainerStarted","Data":"8ee23417e18dd9d150da863ee21a60ee56822aa4fe325768d198602a1c58ed22"}
Oct 30 21:09:59.351616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.351552    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:59.490922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.488359    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:59.621809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.621781    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:59.739614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.739588    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:09:59.766950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.766869    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" event={"ID":"abe73d82-7ffe-44bb-a9fe-ece88010a356","Type":"ContainerStarted","Data":"bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98"}
Oct 30 21:09:59.767315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.767003    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:09:59.849176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:09:59.849132    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" podStartSLOduration=2.443929476 podStartE2EDuration="2.84911759s" totalImagesPullingTime="405.188114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:59.848337143 +0000 UTC m=+582.622737084" watchObservedRunningTime="2026-10-30 21:09:59.84911759 +0000 UTC m=+582.623517528"
Oct 30 21:10:01.094863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:01.094076    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:01.753689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:01.753663    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:10:02.725645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.725613    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:10:02.760425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.760389    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"]
Oct 30 21:10:02.765486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.765457    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:02.784204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.784173    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-ed088d-serving-cert\""
Oct 30 21:10:02.795945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.789162    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"]
Oct 30 21:10:02.916016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.915979    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:02.956568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.956531    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:02.962024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.956608    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:02.962024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.956671    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxd9j\" (UniqueName: \"kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:02.962024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:02.956705    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.046486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.046384    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:03.057741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.057708    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.057944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.057764    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxd9j\" (UniqueName: \"kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.057944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.057797    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.057944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.057909    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.058446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.058421    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.060740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.060717    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.060858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.060721    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.098790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.098754    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxd9j\" (UniqueName: \"kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j\") pod \"maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.151057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.151012    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:03.233967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.233522    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:03.410198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.410163    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:03.485962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.485917    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"]
Oct 30 21:10:03.491864 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:10:03.491805    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c47c92_cdde_40e5_bf9c_7a8762f7fdb7.slice/crio-f2d387eacd9210ec3f8c09fe7b1433ee97f0b7048511aee749680b1c05cd7191 WatchSource:0}: Error finding container f2d387eacd9210ec3f8c09fe7b1433ee97f0b7048511aee749680b1c05cd7191: Status 404 returned error can't find the container with id f2d387eacd9210ec3f8c09fe7b1433ee97f0b7048511aee749680b1c05cd7191
Oct 30 21:10:03.535325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.535296    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:03.768515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.768443    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:10:03.860806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:03.860766    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n" event={"ID":"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7","Type":"ContainerStarted","Data":"f2d387eacd9210ec3f8c09fe7b1433ee97f0b7048511aee749680b1c05cd7191"}
Oct 30 21:10:04.287480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.287437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:04.516310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.516277    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:04.650684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.650597    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:04.737467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.737435    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:10:04.737646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.737521    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:10:04.738288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.738016    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:10:04.870878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.868717    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n" event={"ID":"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7","Type":"ContainerStarted","Data":"e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d"}
Oct 30 21:10:04.870878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.869742    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:04.910384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:04.910278    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n" podStartSLOduration=2.538624613 podStartE2EDuration="2.910257739s" totalImagesPullingTime="371.633126ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:10:04.90797745 +0000 UTC m=+587.682377390" watchObservedRunningTime="2026-10-30 21:10:04.910257739 +0000 UTC m=+587.684657679"
Oct 30 21:10:05.809993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:05.809960    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:10:06.196446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.196394    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:06.884109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.884031    4549 generic.go:356] "Generic (PLEG): container finished" podID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerID="a5e666c7a17afe76d3410711d0c7ce0dae7ad2cdf74e7273e3255702cbcd6db9" exitCode=0
Oct 30 21:10:06.884311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.884105    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" event={"ID":"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99","Type":"ContainerDied","Data":"a5e666c7a17afe76d3410711d0c7ce0dae7ad2cdf74e7273e3255702cbcd6db9"}
Oct 30 21:10:06.885762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.885212    4549 scope.go:122] "RemoveContainer" containerID="a5e666c7a17afe76d3410711d0c7ce0dae7ad2cdf74e7273e3255702cbcd6db9"
Oct 30 21:10:06.894953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.894914    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:06.895078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.894999    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:06.895078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:06.895028    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:07.758227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.758194    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:07.900557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.900527    4549 generic.go:356] "Generic (PLEG): container finished" podID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerID="0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191" exitCode=0
Oct 30 21:10:07.900754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.900604    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" event={"ID":"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83","Type":"ContainerDied","Data":"0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"}
Oct 30 21:10:07.901133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.901113    4549 scope.go:122] "RemoveContainer" containerID="0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"
Oct 30 21:10:07.902533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.902506    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:07.902622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.902576    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:07.902622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.902610    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:07.902782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:07.902765    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" event={"ID":"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99","Type":"ContainerStarted","Data":"91ed29a40c3c258de53ece0a568e4ed5ab4a4e9ab82d448ee49f6e2f9053174a"}
Oct 30 21:10:08.198861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.198455    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:08.386863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.386095    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:10:08.386863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.386238    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:10:08.386863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.386282    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:10:08.816853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.816366    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:08.924455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.924417    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" event={"ID":"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83","Type":"ContainerStarted","Data":"e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf"}
Oct 30 21:10:08.944202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:08.944163    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:09.092927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.091395    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:09.248533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.248498    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:10:09.248712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.248685    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:10:09.248758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.248726    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:10:09.388925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.388366    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:10:09.388925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.388444    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:10:09.774352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:09.774238    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:10.249155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:10.249113    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:10:10.249597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:10.249176    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:10:10.390861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:10.389255    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:10:10.390861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:10.389320    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:10:10.560065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:10.554670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:11.248449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:11.248398    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:10:11.248787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:11.248744    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:10:11.392303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:11.392263    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:10:11.394346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:11.394310    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:10:11.917075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:11.917044    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:10:12.157902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:12.157861    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:12.258371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:12.258282    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:10:12.258655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:12.258618    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:10:12.430728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:12.430210    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:10:12.430728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:12.430271    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:10:13.247763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:13.247728    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:10:13.247986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:13.247788    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 21:10:13.577296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:13.577261    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:10:13.585857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:13.583096    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:10:13.591147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:13.591116    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:10:14.001809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.001706    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.123983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.123931    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.403332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.403295    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:10:14.405969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.405933    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:10:14.406683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.406658    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:10:14.415244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.415213    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.593874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.592653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.720677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:14.720581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:15.042262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:15.042172    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:15.219875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:15.219815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:15.364655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:15.364613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:16.161553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:16.161515    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:17.396770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:17.396746    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:10:17.398928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:17.398908    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:10:17.634204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:17.634170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:19.035527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:19.035486    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:19.040072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:19.040036    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:41194->10.133.0.42:15021: read: connection reset by peer" start-of-body=
Oct 30 21:10:19.040201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:19.040108    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:41194->10.133.0.42:15021: read: connection reset by peer"
Oct 30 21:10:19.337250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:19.337215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:19.546174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:19.546141    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:19.892127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:19.892093    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:20.004175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.004141    4549 generic.go:356] "Generic (PLEG): container finished" podID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerID="ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd" exitCode=0
Oct 30 21:10:20.004355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.004215    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" event={"ID":"0da7f759-7cf9-49d9-b745-b2ac768234a9","Type":"ContainerDied","Data":"ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"}
Oct 30 21:10:20.004629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.004601    4549 scope.go:122] "RemoveContainer" containerID="ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"
Oct 30 21:10:20.005968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.005941    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:20.006054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.006006    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:20.006054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.006039    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:20.024018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.023980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:20.161440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.161342    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:20.834164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:20.834128    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:21.011141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:21.011105    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" event={"ID":"0da7f759-7cf9-49d9-b745-b2ac768234a9","Type":"ContainerStarted","Data":"5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a"}
Oct 30 21:10:21.023521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:21.023481    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:21.692856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:21.692771    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:10:21.693377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:21.692939    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:10:21.693377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:21.693019    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:10:22.584131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:22.584097    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:22.691997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:22.691950    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:10:22.692337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:22.692307    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:10:23.694246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:23.694208    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:10:23.694796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:23.694276    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:10:24.121737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:24.121691    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:24.425675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:24.425593    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:24.691913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:24.691256    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:10:24.691913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:24.691324    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:10:24.727306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:24.727269    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.058804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.058713    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.247548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.247517    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.401710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.401672    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.535332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.535296    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.691360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.691269    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:10:25.691670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.691638    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 21:10:25.891416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:25.891339    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:26.574518 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:10:26.574482    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:10:26.693109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:26.693082    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:10:26.693490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:26.693464    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:10:26.694091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:26.694075    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:10:26.908050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:26.907969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:27.639947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:27.639911    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:29.113946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:29.113896    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:30.538150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:30.538111    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:32.103546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:32.103499    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:33.620113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:33.620061    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:35.106104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:35.106065    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:35.329093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:35.329057    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:36.490592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:36.490553    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:37.899673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:37.899642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:38.747703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:38.747670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:39.338433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:39.338400    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:40.862424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:40.862391    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:41.349020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:41.348981    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:41.531191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:41.531159    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:42.403358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:42.403307    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:43.808219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:43.808187    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:45.207745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:45.207710    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:46.674577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:46.674543    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:47.013573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:47.013492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:48.209132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:48.209094    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:49.580958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:49.580928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:51.042485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:51.042455    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:51.384571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:51.384537    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:52.589777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:52.589735    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:54.001941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:54.001901    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:55.388680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:55.388639    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:56.575147 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:10:56.575106    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:10:56.899037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:56.898938    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:58.303069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:58.303034    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:59.826016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:10:59.825964    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:01.309618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:01.309582    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:01.438161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:01.438102    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:02.805792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:02.805751    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.303067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:04.303029    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.411117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:04.411079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.955181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:05.955146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:06.088141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:06.088106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:06.222793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:06.222704    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:07.542249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:07.542216    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.135763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:09.135721    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.633662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:10.633623    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:11.266752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:11.266718    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:11.400150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:11.400105    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.128090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:12.128059    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:13.630752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:13.630717    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:15.112519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:15.112486    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:15.849116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:15.849071    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.864603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:16.864568    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:18.292849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:18.292807    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:19.153275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.153241    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"]
Oct 30 21:11:19.156772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.156751    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.183924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.183896    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-58fea9\"/\"kube-root-ca.crt\""
Oct 30 21:11:19.183924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.183899    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-58fea9\"/\"openshift-service-ca.crt\""
Oct 30 21:11:19.184129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.183980    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-58fea9\"/\"premium-w0-58fea9-kserve-self-signed-certs\""
Oct 30 21:11:19.184129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.184039    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-58fea9\"/\"default-dockercfg-wmzx2\""
Oct 30 21:11:19.186084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.186054    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"]
Oct 30 21:11:19.242709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.242674    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:19.250177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.250146    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.250340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.250198    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.250340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.250320    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.250477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.250461    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.250559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.250538    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvc9f\" (UniqueName: \"kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.250597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.250574    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.352464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.352330    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.353941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.353900    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvc9f\" (UniqueName: \"kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.354150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.354133    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.354288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.354273    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.354416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.354404    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.354530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.354516    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.358277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.357863    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.358277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.353490    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.377568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.377538    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.388851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.386894    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.397004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.393868    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.399621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.399584    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvc9f\" (UniqueName: \"kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f\") pod \"premium-w0-58fea9-kserve-8698dc969b-8hf4l\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") " pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.470029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.469943    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:19.769936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.769908    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"]
Oct 30 21:11:19.771484 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:11:19.771463    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9ee38ff_372c_424c_892b_a3b4880e0c34.slice/crio-3cf78f527f9971f4285dd182e40c290bda6200918c89355d3a5d4a753839e658 WatchSource:0}: Error finding container 3cf78f527f9971f4285dd182e40c290bda6200918c89355d3a5d4a753839e658: Status 404 returned error can't find the container with id 3cf78f527f9971f4285dd182e40c290bda6200918c89355d3a5d4a753839e658
Oct 30 21:11:19.810718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:19.810691    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:20.312287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:20.312246    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" event={"ID":"b9ee38ff-372c-424c-892b-a3b4880e0c34","Type":"ContainerStarted","Data":"04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d"}
Oct 30 21:11:20.312510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:20.312297    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" event={"ID":"b9ee38ff-372c-424c-892b-a3b4880e0c34","Type":"ContainerStarted","Data":"3cf78f527f9971f4285dd182e40c290bda6200918c89355d3a5d4a753839e658"}
Oct 30 21:11:20.351337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:20.351284    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podStartSLOduration=1.351264202 podStartE2EDuration="1.351264202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:20.349370287 +0000 UTC m=+663.123770228" watchObservedRunningTime="2026-10-30 21:11:20.351264202 +0000 UTC m=+663.125664147"
Oct 30 21:11:20.478444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:20.475843    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:21.209298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:21.209264    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:22.617130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:22.617091    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:23.449300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:23.449267    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.022813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:24.022771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.517303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:25.517272    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:26.573935 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:11:26.573894    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:11:27.222539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:27.222507    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:28.769941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:28.769808    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:29.470945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:29.470907    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:29.483770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:29.483745    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:30.234557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:30.234504    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.363463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:30.363431    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.376788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:30.376759    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:11:31.823382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:31.823349    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:33.225207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:33.225169    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:34.707318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:34.707274    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.132778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:36.132726    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:37.739585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:37.739548    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.383503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:39.383455    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:40.930884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:40.930816    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.518383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:42.518348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.034769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:44.034736    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:45.038767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.038728    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"]
Oct 30 21:11:45.041816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.041795    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.062295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.062261    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"]
Oct 30 21:11:45.068136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.068109    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-58fea9-openshift-default-dockercfg-2wlz5\""
Oct 30 21:11:45.142101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142062    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142103    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142128    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142275    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142434    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142482    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142537    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142597    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.142709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.142671    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd6gv\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.243769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243733    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.243963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243859    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.243963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243885    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.243963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243902    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.243963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243922    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.243963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243943    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd6gv\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.243982    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244016    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244047    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244197    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244369    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244399    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244404    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.244947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.244928    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.246425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.246399    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.246591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.246570    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.268596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.268557    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.269097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.269074    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd6gv\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.356048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.356012    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:45.575951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.575909    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"]
Oct 30 21:11:45.578732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.578713    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.596660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.596635    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-79477e-openshift-default-dockercfg-448fh\""
Oct 30 21:11:45.605963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.605936    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"]
Oct 30 21:11:45.647930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.647848    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.647930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.647904    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648010    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648038    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648063    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648080    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648183    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648231    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ths4d\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.648337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.648299    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.677337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.677307    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"]
Oct 30 21:11:45.679495 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:11:45.679458    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5867ad60_94d0_4054_8649_135537d67730.slice/crio-dcac1fb31e72281fd68b87d6510d4a775037e17ae19ced909729011d41bde41f WatchSource:0}: Error finding container dcac1fb31e72281fd68b87d6510d4a775037e17ae19ced909729011d41bde41f: Status 404 returned error can't find the container with id dcac1fb31e72281fd68b87d6510d4a775037e17ae19ced909729011d41bde41f
Oct 30 21:11:45.682226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.682190    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:11:45.682326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.682279    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:11:45.682406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.682324    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:11:45.714353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.714284    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750122    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750292    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750369    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750465    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750525    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750584    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750612    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750680    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.750766    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ths4d\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.752518    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.753249    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.753499    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.753851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.753740    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.756132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.756084    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.761116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.760283    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.763606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.763576    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.771570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.771539    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ths4d\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.774957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.774932    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:45.893424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:45.893386    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:46.208542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.208506    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"]
Oct 30 21:11:46.209734 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:11:46.209711    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203335d3_dfdf_40dc_8936_db10cb422b4e.slice/crio-4d84900d56cf1a7d2e3ecda6de1dabeb2a2279b3657c96379c7641cbb22aafdc WatchSource:0}: Error finding container 4d84900d56cf1a7d2e3ecda6de1dabeb2a2279b3657c96379c7641cbb22aafdc: Status 404 returned error can't find the container with id 4d84900d56cf1a7d2e3ecda6de1dabeb2a2279b3657c96379c7641cbb22aafdc
Oct 30 21:11:46.212201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.212168    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:11:46.212304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.212230    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:11:46.212304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.212259    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:11:46.451885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.451847    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" event={"ID":"203335d3-dfdf-40dc-8936-db10cb422b4e","Type":"ContainerStarted","Data":"b96cbda3c5a1db6577174d82f68e51a431a9ad250103b4886df3e66f556f664d"}
Oct 30 21:11:46.452083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.451917    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" event={"ID":"203335d3-dfdf-40dc-8936-db10cb422b4e","Type":"ContainerStarted","Data":"4d84900d56cf1a7d2e3ecda6de1dabeb2a2279b3657c96379c7641cbb22aafdc"}
Oct 30 21:11:46.453568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.453537    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" event={"ID":"5867ad60-94d0-4054-8649-135537d67730","Type":"ContainerStarted","Data":"dadd5b6bda2e5104c26408bf21ee18763ac80acda4a5e7348d8425bb0484600f"}
Oct 30 21:11:46.453697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.453597    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" event={"ID":"5867ad60-94d0-4054-8649-135537d67730","Type":"ContainerStarted","Data":"dcac1fb31e72281fd68b87d6510d4a775037e17ae19ced909729011d41bde41f"}
Oct 30 21:11:46.496917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.496669    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podStartSLOduration=1.496646817 podStartE2EDuration="1.496646817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:46.494301473 +0000 UTC m=+689.268701415" watchObservedRunningTime="2026-10-30 21:11:46.496646817 +0000 UTC m=+689.271046758"
Oct 30 21:11:46.540312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.540220    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podStartSLOduration=2.540197496 podStartE2EDuration="2.540197496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:46.537390825 +0000 UTC m=+689.311790763" watchObservedRunningTime="2026-10-30 21:11:46.540197496 +0000 UTC m=+689.314597436"
Oct 30 21:11:46.898856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:46.897884    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:47.356623 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:47.356592    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:47.358386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:47.358360    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 21:11:47.358513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:47.358411    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 21:11:47.365298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:47.365168    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.902005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:47.901962    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:11:47.902307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:47.902273    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:11:48.356311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:48.356271    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 21:11:48.356521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:48.356322    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 21:11:48.897335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:48.897293    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:11:48.898442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:48.898025    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:11:49.302817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:49.302726    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:49.357691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:49.356516    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 21:11:49.357691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:49.356574    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 21:11:49.895290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:49.895247    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:11:49.895663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:49.895620    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:11:50.356309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:50.356278    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 21:11:50.356762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:50.356332    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 21:11:50.894738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:50.894264    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:11:50.894738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:50.894329    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:11:51.305917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:51.305673    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:51.358103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:51.358071    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 21:11:51.359107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:51.358685    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 21:11:51.896095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:51.896057    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:11:51.896415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:51.896381    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:11:52.358349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:52.358304    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 21:11:52.359045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:52.358985    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 21:11:53.045378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.045337    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:53.200118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.200081    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:53.201054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.201027    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:53.202386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.202361    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:11:53.454492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.454456    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:53.455027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.454886    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:53.456803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:53.456782    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:11:54.730353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:54.730291    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:56.418592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:56.418562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:56.572575 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:11:56.572533    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:11:57.822047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:57.822017    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:59.217073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:11:59.217039    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:00.264345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.264310    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"]
Oct 30 21:12:00.269157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.269136    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.293819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.293792    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-43c070-openshift-default-dockercfg-srjfd\""
Oct 30 21:12:00.312748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.312711    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"]
Oct 30 21:12:00.370296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370256    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-289xz\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370331    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370384    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370412    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370485    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370567    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370591    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370631    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.370878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.370714    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.471685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.471647    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.471890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.471730    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.471890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.471763    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.471890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.471806    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.471939    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472042    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472125    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472213    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472267    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-289xz\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472527    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.472776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472730    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.473017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.472991    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.473189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.473159    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.473305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.473173    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.474785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.474758    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.474952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.474935    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.491934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.491904    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.492053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.492038    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-289xz\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.582946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.582669    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:00.839735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.839661    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:00.906499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.906468    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"]
Oct 30 21:12:00.911096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.911072    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"]
Oct 30 21:12:00.911240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.911216    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:00.912516 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:12:00.912490    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ffaa97_07b5_4419_b324_f263db44366c.slice/crio-516c319fbaedae1d0a558c38a69ff5a1fd67a4a54119fee73c252b7685c9f706 WatchSource:0}: Error finding container 516c319fbaedae1d0a558c38a69ff5a1fd67a4a54119fee73c252b7685c9f706: Status 404 returned error can't find the container with id 516c319fbaedae1d0a558c38a69ff5a1fd67a4a54119fee73c252b7685c9f706
Oct 30 21:12:00.915249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.915218    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:00.915333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.915297    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:00.915333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.915327    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:00.925942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.925917    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ed088d-openshift-default-dockercfg-xqfc5\""
Oct 30 21:12:00.933920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:00.933897    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"]
Oct 30 21:12:01.080332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080301    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080351    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080461    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080480    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080559    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080628    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080655    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdjf\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080682    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.080861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.080735    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181573    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181612    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181646    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181710    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181738    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdjf\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181764    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181799    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181909    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.181983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.181966    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.182295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.182086    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.182295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.182117    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.182403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.182331    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.182750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.182728    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.182750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.182755    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.184748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.184726    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.184870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.184855    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.201505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.201477    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.201856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.201814    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdjf\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.228334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.228303    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:01.548040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.548014    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"]
Oct 30 21:12:01.549868 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:12:01.549804    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15bb9ec9_80f1_48e3_a32d_f3ff9c3d6399.slice/crio-1cf3611a8b3c93a49bd1925306139f7a7708438f8333d19246e965a2045ec5be WatchSource:0}: Error finding container 1cf3611a8b3c93a49bd1925306139f7a7708438f8333d19246e965a2045ec5be: Status 404 returned error can't find the container with id 1cf3611a8b3c93a49bd1925306139f7a7708438f8333d19246e965a2045ec5be
Oct 30 21:12:01.552429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.552396    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:01.552503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.552460    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:01.552503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.552488    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:01.625749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.625689    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" event={"ID":"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399","Type":"ContainerStarted","Data":"1cf3611a8b3c93a49bd1925306139f7a7708438f8333d19246e965a2045ec5be"}
Oct 30 21:12:01.629570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.628838    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" event={"ID":"d7ffaa97-07b5-4419-b324-f263db44366c","Type":"ContainerStarted","Data":"c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042"}
Oct 30 21:12:01.629570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:01.628902    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" event={"ID":"d7ffaa97-07b5-4419-b324-f263db44366c","Type":"ContainerStarted","Data":"516c319fbaedae1d0a558c38a69ff5a1fd67a4a54119fee73c252b7685c9f706"}
Oct 30 21:12:02.396858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.393681    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:02.583377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.583339    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:02.595911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.589196    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:12:02.595911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.589276    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:12:02.640872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.640451    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" event={"ID":"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399","Type":"ContainerStarted","Data":"2e3b2433d042239dbfcf779e364e078152f9b81f8a05ea87ce1bf311dbe3b21b"}
Oct 30 21:12:02.716015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.710277    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podStartSLOduration=2.710258284 podStartE2EDuration="2.710258284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:02.697320674 +0000 UTC m=+705.471720639" watchObservedRunningTime="2026-10-30 21:12:02.710258284 +0000 UTC m=+705.484658225"
Oct 30 21:12:02.716015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:02.710573    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podStartSLOduration=2.710563633 podStartE2EDuration="2.710563633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:01.698167524 +0000 UTC m=+704.472567464" watchObservedRunningTime="2026-10-30 21:12:02.710563633 +0000 UTC m=+705.484963573"
Oct 30 21:12:03.229542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:03.229509    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:03.230818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:03.230795    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:12:03.230926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:03.230890    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:12:03.583852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:03.583250    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:12:03.583852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:03.583331    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:12:04.238857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:04.229309    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:12:04.238857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:04.229377    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:12:04.406857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:04.406774    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.571470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:04.571367    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.585336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:04.585293    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:12:04.586404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:04.586369    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:12:05.229320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:05.229290    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:12:05.229581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:05.229558    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:12:05.586856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:05.585574    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:12:05.586856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:05.585649    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:12:06.228891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:06.228818    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:12:06.229076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:06.228904    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:12:06.320923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:06.320887    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:06.443170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:06.443128    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:06.586876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:06.586259    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:12:06.586876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:06.586338    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:12:07.229201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:07.229160    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:12:07.229391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:07.229222    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:12:07.584855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:07.584157    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 21:12:07.584855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:07.584241    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 21:12:07.912188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:07.912096    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:08.229682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:08.229590    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:12:08.229682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:08.229653    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:12:08.588772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:08.588740    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:08.589155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:08.589130    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:08.589860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:08.589816    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:12:09.233633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:09.233600    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:09.234090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:09.234051    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:09.234964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:09.234941    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:12:09.701048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:09.701018    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.197631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:11.197594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:13.033294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:13.033255    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:14.664272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:14.664238    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:16.221266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:16.221235    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.693784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:17.693752    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:19.146800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:19.146757    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:20.571398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:20.571364    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:22.115878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:22.115844    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:23.632318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:23.632286    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:25.177758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:25.177715    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.573927 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:12:26.573892    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:12:26.728889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:26.728820    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.240296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:28.240259    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:29.910406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:29.910362    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:31.546346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:31.546303    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.336151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:32.336111    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.463407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:32.463369    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:33.044244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:33.044197    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:34.529067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:34.529030    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:36.018530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:36.018492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:38.901491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:38.901453    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.491616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:40.491584    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.822858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:40.822799    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:42.013464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:42.013433    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:43.598120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:43.598087    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:45.209550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.209521    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:45.585881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.585844    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"]
Oct 30 21:12:45.590540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.590523    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.622107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.622070    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"]
Oct 30 21:12:45.764108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764068    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764170    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764270    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764309    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764340    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764438    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764462    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764506    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.764650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.764601    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46z2l\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866016    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46z2l\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866067    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866101    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866178    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866228    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866267    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866335    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866366    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866430    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866652    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866671    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.866967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.866944    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.867217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.867186    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.867388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.867370    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.868686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.868661    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.868916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.868900    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.885200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.885174    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.886096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.886080    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46z2l\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:45.907249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:45.907216    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:46.225297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:46.225200    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"]
Oct 30 21:12:46.229908 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:12:46.229876    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d490321_2533_40fc_a4e8_a90521b435b8.slice/crio-9b7eca8e6b4150a6d9f1104d1e87db511e2978ab949625bd14924711679a38d4 WatchSource:0}: Error finding container 9b7eca8e6b4150a6d9f1104d1e87db511e2978ab949625bd14924711679a38d4: Status 404 returned error can't find the container with id 9b7eca8e6b4150a6d9f1104d1e87db511e2978ab949625bd14924711679a38d4
Oct 30 21:12:46.233174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:46.233130    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:46.233293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:46.233215    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:46.233293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:46.233258    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:46.846284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:46.846243    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:47.026101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.026062    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" event={"ID":"8d490321-2533-40fc-a4e8-a90521b435b8","Type":"ContainerStarted","Data":"a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05"}
Oct 30 21:12:47.026101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.026121    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" event={"ID":"8d490321-2533-40fc-a4e8-a90521b435b8","Type":"ContainerStarted","Data":"9b7eca8e6b4150a6d9f1104d1e87db511e2978ab949625bd14924711679a38d4"}
Oct 30 21:12:47.099768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.099659    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podStartSLOduration=2.099644526 podStartE2EDuration="2.099644526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:47.08638512 +0000 UTC m=+749.860785059" watchObservedRunningTime="2026-10-30 21:12:47.099644526 +0000 UTC m=+749.874044465"
Oct 30 21:12:47.632857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.632479    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:47.916774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.916042    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:47.916774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.916212    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:12:47.916774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:47.916273    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:12:48.131856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.131662    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"]
Oct 30 21:12:48.145858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.144116    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.166102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.166066    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"]
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.320932    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321051    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321105    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321139    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321257    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stdzs\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321289    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321399    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321433    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.369856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.321503    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.429854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.429779    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.430063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.429943    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.430063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.429974    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.430247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.430138    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stdzs\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.430451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.430434    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.431481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.431460    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.431746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.431731    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.432087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.431982    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.432368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.432353    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.453866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.433482    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.453866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.433746    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.453866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.434374    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.459083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.459000    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.461168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.460137    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.461168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.460960    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.497456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.492317    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.514419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.511729    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stdzs\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.535865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.530500    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-m9l2r\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.684501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.684192    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:48.908858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.908341    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:12:48.908858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:48.908417    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:12:49.647618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.634001    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:49.647618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.642948    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:49.647618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.643040    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:49.647618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.643084    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:12:49.660189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.660150    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"]
Oct 30 21:12:49.663588 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:12:49.663548    4549 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d4aa36_2d5e_463b_9b7d_e7825c2abe0f.slice/crio-3472653bacea894d53a40298d61146631cc8a393f62e5c0ed3ed828c8d9fbaba\": RecentStats: unable to find data in memory cache]"
Oct 30 21:12:49.910940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.908215    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:12:49.910940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:49.908279    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:12:50.065058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.065014    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" event={"ID":"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f","Type":"ContainerStarted","Data":"7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300"}
Oct 30 21:12:50.065244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.065101    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" event={"ID":"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f","Type":"ContainerStarted","Data":"3472653bacea894d53a40298d61146631cc8a393f62e5c0ed3ed828c8d9fbaba"}
Oct 30 21:12:50.690855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.690408    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:50.690855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.690544    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:50.690855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.690590    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:50.914856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.914115    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:12:50.914856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:50.914176    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:12:51.260702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:51.257962    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:51.691856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:51.688315    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:51.691856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:51.688380    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:51.910856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:51.910430    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:12:51.910856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:51.910489    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:12:52.689292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:52.689252    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:52.689955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:52.689916    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:52.910322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:52.910280    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:12:52.910507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:52.910350    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:12:53.154949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:53.154911    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:53.687855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:53.687236    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:53.687855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:53.687298    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:54.691856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.691233    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:54.691856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.691298    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:54.819434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.819402    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:54.819717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.819696    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:54.820619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.820602    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:12:54.880857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.878759    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podStartSLOduration=6.878741752 podStartE2EDuration="6.878741752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:50.114185657 +0000 UTC m=+752.888585598" watchObservedRunningTime="2026-10-30 21:12:54.878741752 +0000 UTC m=+757.653141693"
Oct 30 21:12:54.902971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:54.902928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:55.692396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:55.692357    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:55.693524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:55.693073    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:56.504978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:56.504945    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:56.605206 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:12:56.605173    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:12:56.686030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:56.685996    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:56.686233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:56.686052    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:57.685724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:57.685678    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:12:57.686164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:57.685752    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:12:58.007370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:58.007271    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:58.690134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:58.690063    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:58.690530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:58.690350    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:58.690939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:58.690922    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:12:59.507477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:12:59.507444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:00.643734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.643689    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"]
Oct 30 21:13:00.650044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.650013    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.680607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.680573    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"]
Oct 30 21:13:00.710520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710476    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710547    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710631    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710684    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710712    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77tj9\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710792    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710845    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.710989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.710889    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.711079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.711008    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812166    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812212    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77tj9\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812327    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812375    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812411    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812471    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812542    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812582    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812631    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812707    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812751    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.812892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812810    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.813009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.812990    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.813074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.813019    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.814949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.814924    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.815235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.815217    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.831691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.831663    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77tj9\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.832184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.832165    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:00.963627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:00.963537    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:01.000048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:01.000016    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:01.279323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:01.279214    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"]
Oct 30 21:13:01.285314 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:01.285268    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b363bb0_0398_4bd5_84f0_5949a491cd35.slice/crio-86de8d2ab9450b5ad8a994ae0b0caf76d5fdc0130fc4588372c2cca55814c381 WatchSource:0}: Error finding container 86de8d2ab9450b5ad8a994ae0b0caf76d5fdc0130fc4588372c2cca55814c381: Status 404 returned error can't find the container with id 86de8d2ab9450b5ad8a994ae0b0caf76d5fdc0130fc4588372c2cca55814c381
Oct 30 21:13:01.287938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:01.287893    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:01.288029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:01.287985    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:01.288077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:01.288030    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:01.346215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:01.346179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:02.162207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.162176    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" event={"ID":"6b363bb0-0398-4bd5-84f0-5949a491cd35","Type":"ContainerStarted","Data":"73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e"}
Oct 30 21:13:02.162635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.162225    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" event={"ID":"6b363bb0-0398-4bd5-84f0-5949a491cd35","Type":"ContainerStarted","Data":"86de8d2ab9450b5ad8a994ae0b0caf76d5fdc0130fc4588372c2cca55814c381"}
Oct 30 21:13:02.221562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.221508    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podStartSLOduration=2.221493726 podStartE2EDuration="2.221493726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:02.220725514 +0000 UTC m=+764.995125453" watchObservedRunningTime="2026-10-30 21:13:02.221493726 +0000 UTC m=+764.995893665"
Oct 30 21:13:02.661174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.661138    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:02.964772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.964683    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:02.966117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.966093    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:13:02.966224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:02.966138    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:13:03.964447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:03.964414    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:13:03.964919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:03.964465    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:13:04.242458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:04.242352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:04.709960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:04.709920    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:04.964521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:04.964412    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:13:04.964521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:04.964468    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:13:05.965856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:05.965256    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:13:05.965856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:05.965322    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:13:06.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:06.170772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:06.965857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:06.964307    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:13:06.965857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:06.964388    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:13:07.048874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.045381    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"]
Oct 30 21:13:07.051244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.051217    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.055818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.055794    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.055999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.055979    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.056158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.056138    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.056239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.056208    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.056294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.056248    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.056343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.056288    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvlv9\" (UniqueName: \"kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.084342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.084213    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"]
Oct 30 21:13:07.099886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.091245    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-2cda44\"/\"distinct-w2-2cda44-kserve-self-signed-certs\""
Oct 30 21:13:07.099886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.091986    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-2cda44\"/\"openshift-service-ca.crt\""
Oct 30 21:13:07.099886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.092095    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-2cda44\"/\"default-dockercfg-qz7t2\""
Oct 30 21:13:07.099886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.092180    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-2cda44\"/\"kube-root-ca.crt\""
Oct 30 21:13:07.128091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.127897    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:07.157285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.157253    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.157465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.157341    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.157465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.157380    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.157465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.157418    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.157465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.157441    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.157465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.157462    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvlv9\" (UniqueName: \"kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.158365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.158033    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.158365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.158196    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.158365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.158342    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.160563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.160350    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.161436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.161418    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.199670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.199630    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvlv9\" (UniqueName: \"kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9\") pod \"distinct-w2-2cda44-kserve-68468959c9-2tjhb\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") " pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.411858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.410214    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:07.744784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.744757    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"]
Oct 30 21:13:07.746072 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:07.746047    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c4e39e9_43c4_42d5_bf91_e0b660d634c7.slice/crio-1740d49fc9dd54d9b35e89d3b080b59b07323be7c39ad0081c905c2d3f8658dd WatchSource:0}: Error finding container 1740d49fc9dd54d9b35e89d3b080b59b07323be7c39ad0081c905c2d3f8658dd: Status 404 returned error can't find the container with id 1740d49fc9dd54d9b35e89d3b080b59b07323be7c39ad0081c905c2d3f8658dd
Oct 30 21:13:07.964667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.964630    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:13:07.964938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:07.964696    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:13:08.021011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.020909    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:08.241224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.241182    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" event={"ID":"5c4e39e9-43c4-42d5-bf91-e0b660d634c7","Type":"ContainerStarted","Data":"2b8db577d7a6e85ce58032de620a49c1b773924d5c0cf56522d857f8859748e2"}
Oct 30 21:13:08.241406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.241238    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" event={"ID":"5c4e39e9-43c4-42d5-bf91-e0b660d634c7","Type":"ContainerStarted","Data":"1740d49fc9dd54d9b35e89d3b080b59b07323be7c39ad0081c905c2d3f8658dd"}
Oct 30 21:13:08.282672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.282554    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podStartSLOduration=1.282533219 podStartE2EDuration="1.282533219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:08.279270372 +0000 UTC m=+771.053670322" watchObservedRunningTime="2026-10-30 21:13:08.282533219 +0000 UTC m=+771.056933161"
Oct 30 21:13:08.415032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.414982    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:08.969010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.968973    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:08.969271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.969250    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:08.970056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:08.970038    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:13:09.056043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:09.056012    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:09.719857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:09.719791    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.224346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:11.224310    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:12.704191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:12.704159    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:14.227592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:14.227558    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:15.589975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.589938    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"]
Oct 30 21:13:15.594304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.594281    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.621317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.621286    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"]
Oct 30 21:13:15.657388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657350    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657455    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657489    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657616    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657659    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657733    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77tq4\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657799    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657859    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.657947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.657901    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.758907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.758863    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.758925    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.758986    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759019    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759092    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759118    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759178    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77tq4\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759256    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759303    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759320    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759442    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759625    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.759854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.759817    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.761473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.761451    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.761564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.761486    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.778581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.778550    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77tq4\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.778739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.778620    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:15.778739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.778688    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:15.922022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:15.921932    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:16.177495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.177405    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"]
Oct 30 21:13:16.182518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.182493    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.204766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.204731    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"]
Oct 30 21:13:16.257276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.257250    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"]
Oct 30 21:13:16.258860 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:16.258799    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060dbc88_01cf_4f36_8f18_2cf414a32c2a.slice/crio-a3174819611e66682b0cf015fe028a2a523256e022d47e75db962c784333f5a4 WatchSource:0}: Error finding container a3174819611e66682b0cf015fe028a2a523256e022d47e75db962c784333f5a4: Status 404 returned error can't find the container with id a3174819611e66682b0cf015fe028a2a523256e022d47e75db962c784333f5a4
Oct 30 21:13:16.261274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.261241    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:16.261354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.261305    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:16.261354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.261332    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:16.265711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.265688    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzbg\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.265817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.265736    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.265817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.265760    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.265933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.265848    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.265933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.265903    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.266001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.265979    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.266033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.266013    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.266063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.266046    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.266097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.266076    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.338813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.338777    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" event={"ID":"060dbc88-01cf-4f36-8f18-2cf414a32c2a","Type":"ContainerStarted","Data":"fa33cbeadd7b7d0a21e7e369f7ba82e88fb4ef868af26762a19e55f9a36560d7"}
Oct 30 21:13:16.338987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.338851    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" event={"ID":"060dbc88-01cf-4f36-8f18-2cf414a32c2a","Type":"ContainerStarted","Data":"a3174819611e66682b0cf015fe028a2a523256e022d47e75db962c784333f5a4"}
Oct 30 21:13:16.367114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367071    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367139    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367172    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367256    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367288    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367334    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367388    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367449    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.367640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.367634    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzbg\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.368658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.368416    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.368658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.368476    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.368761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.368659    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.369653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.369627    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.369873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.369855    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.372334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.372305    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.372480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.372457    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.380987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.380930    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podStartSLOduration=1.380914043 podStartE2EDuration="1.380914043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:16.379851558 +0000 UTC m=+779.154251494" watchObservedRunningTime="2026-10-30 21:13:16.380914043 +0000 UTC m=+779.155313984"
Oct 30 21:13:16.387571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.387526    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzbg\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.391185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.391160    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.499488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.499396    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:16.822176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.822147    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"]
Oct 30 21:13:16.823356 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:16.823329    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040183c8_21bd_44d3_bde2_bc99b6e3bc9f.slice/crio-e154f9a4d1961c5c019117e21949a2a999c23a0cd5b02429992ede27467f7402 WatchSource:0}: Error finding container e154f9a4d1961c5c019117e21949a2a999c23a0cd5b02429992ede27467f7402: Status 404 returned error can't find the container with id e154f9a4d1961c5c019117e21949a2a999c23a0cd5b02429992ede27467f7402
Oct 30 21:13:16.825913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.825878    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:16.826014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.825971    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:16.826082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.826016    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:16.924350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:16.924308    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:17.299922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.292916    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:17.428681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.428517    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:17.430091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.430064    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" event={"ID":"040183c8-21bd-44d3-bde2-bc99b6e3bc9f","Type":"ContainerStarted","Data":"a452281bf5a452da8ad1f21969011ee97f99dd4712253412a0e2da00030e5912"}
Oct 30 21:13:17.430285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.430269    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" event={"ID":"040183c8-21bd-44d3-bde2-bc99b6e3bc9f","Type":"ContainerStarted","Data":"e154f9a4d1961c5c019117e21949a2a999c23a0cd5b02429992ede27467f7402"}
Oct 30 21:13:17.474856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.460319    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:17.522858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.504884    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:17.522858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.505055    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:17.522858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.505104    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:17.522858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.511275    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podStartSLOduration=1.511256371 podStartE2EDuration="1.511256371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:17.495542989 +0000 UTC m=+780.269942928" watchObservedRunningTime="2026-10-30 21:13:17.511256371 +0000 UTC m=+780.285656309"
Oct 30 21:13:17.937907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.930254    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:17.937907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:17.930470    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:18.510889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:18.509920    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:18.510889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:18.510036    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:18.924229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:18.924190    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:18.924526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:18.924500    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:19.389854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.389790    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:13:19.391722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.390433    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:13:19.425553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.425520    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:13:19.499915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.499878    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:19.500185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.500156    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:19.951857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.924237    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:19.974855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:19.966920    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:20.125857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.124762    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:20.404851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.403901    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:13:20.404851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.404582    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:13:20.456890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.425949    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:13:20.479242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.479204    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:13:20.518706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.518674    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:20.519207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.519166    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:20.925921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.923369    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:20.925921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.923423    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:20.930616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:20.930596    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:13:21.510857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:21.509482    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:21.510857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:21.509550    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:21.651185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:21.651152    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:21.937119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:21.937073    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:21.937438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:21.937394    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:22.020844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:22.020776    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.502306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:22.502262    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:22.502628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:22.502600    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:22.923177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:22.923144    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:22.923597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:22.923202    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:23.503270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:23.503230    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:23.503582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:23.503534    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:23.765187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:23.765089    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:23.922810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:23.922773    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:13:23.923053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:23.922874    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:13:24.500127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:24.500083    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:13:24.500574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:24.500145    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:13:25.242096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.242039    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:25.242752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.242723    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:25.244499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.244476    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:13:25.302394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.302133    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:25.910206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.910169    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:25.910740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.910714    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:25.912259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:25.912228    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:13:26.574367 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:13:26.574327    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:13:26.803793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:26.803753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:28.305739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:28.305703    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:29.811936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:29.811901    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:31.411594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:31.411558    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:32.833531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:32.833500    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:33.607326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.607293    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"]
Oct 30 21:13:33.611390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.611366    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.628656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.626818    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.628656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.626969    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.628656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.627016    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.628656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.627069    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.628656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.627100    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.628656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.627214    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frmsb\" (UniqueName: \"kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.630515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.630488    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"]
Oct 30 21:13:33.639100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.639074    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-79477e\"/\"kube-root-ca.crt\""
Oct 30 21:13:33.640953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.640927    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-79477e\"/\"default-dockercfg-74c78\""
Oct 30 21:13:33.641076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.640959    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-79477e\"/\"distinct-2-w3-79477e-kserve-self-signed-certs\""
Oct 30 21:13:33.641076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.640981    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-79477e\"/\"openshift-service-ca.crt\""
Oct 30 21:13:33.698521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.698484    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:33.727993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.727953    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728007    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728056    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728082    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728172    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frmsb\" (UniqueName: \"kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728253    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728369    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728408    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.728564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.728474    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.730456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.730434    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.730724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.730707    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.746756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.746732    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frmsb\" (UniqueName: \"kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb\") pod \"distinct-2-w3-79477e-kserve-59b74cb58d-wljks\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") " pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:33.926189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:33.926087    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:34.258903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:34.258864    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"]
Oct 30 21:13:34.259181 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:34.259160    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff1362cb_2c95_45a4_8a59_6c7a2d335192.slice/crio-0e3b0175ebbdf6a3915404758a5593ca53210a578ca69bac62e8f9a32ca8465e WatchSource:0}: Error finding container 0e3b0175ebbdf6a3915404758a5593ca53210a578ca69bac62e8f9a32ca8465e: Status 404 returned error can't find the container with id 0e3b0175ebbdf6a3915404758a5593ca53210a578ca69bac62e8f9a32ca8465e
Oct 30 21:13:34.416507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:34.416472    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:35.049742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:35.049698    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" event={"ID":"ff1362cb-2c95-45a4-8a59-6c7a2d335192","Type":"ContainerStarted","Data":"d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36"}
Oct 30 21:13:35.049742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:35.049743    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" event={"ID":"ff1362cb-2c95-45a4-8a59-6c7a2d335192","Type":"ContainerStarted","Data":"0e3b0175ebbdf6a3915404758a5593ca53210a578ca69bac62e8f9a32ca8465e"}
Oct 30 21:13:35.090141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:35.090077    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podStartSLOduration=2.090061655 podStartE2EDuration="2.090061655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:35.087714013 +0000 UTC m=+797.862113952" watchObservedRunningTime="2026-10-30 21:13:35.090061655 +0000 UTC m=+797.864461593"
Oct 30 21:13:35.911596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:35.911563    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:35.927654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:35.927614    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:37.410447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:37.410413    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:39.232984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:39.232946    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:40.791856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:40.790902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:41.283250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:41.283219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:42.315098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:42.315055    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:43.719465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:43.719432    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:43.927678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:43.927643    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:43.940797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:43.940772    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:44.118663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:44.118625    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:13:45.224367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.224327    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:45.398300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.398268    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"]
Oct 30 21:13:45.403339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.403317    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.434818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.434781    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"]
Oct 30 21:13:45.526206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526124    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526159    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526179    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526266    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526361    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526393    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526501    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jshmz\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526638    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.526711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.526696    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.627739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627708    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.627943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627764    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.627943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627801    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.627943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627818    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.627943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627867    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.627943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627895    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.627971    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628013    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628053    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jshmz\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628198    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628264    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628344    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628354    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.628813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.628795    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.630370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.630353    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.630461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.630420    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.648173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.648146    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.648338 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.648319    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jshmz\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:45.717970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:45.717943    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:46.033787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.033759    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"]
Oct 30 21:13:46.034365 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:46.034342    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8f2f70_bf79_4576_ae89_9f43a1713e5c.slice/crio-2f03ecf27d543e26840304ad81a843d76a4a8eaba4717b812c744e8179919f9d WatchSource:0}: Error finding container 2f03ecf27d543e26840304ad81a843d76a4a8eaba4717b812c744e8179919f9d: Status 404 returned error can't find the container with id 2f03ecf27d543e26840304ad81a843d76a4a8eaba4717b812c744e8179919f9d
Oct 30 21:13:46.036648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.036617    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:46.036743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.036679    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:46.036743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.036707    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:46.119705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.119668    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" event={"ID":"7b8f2f70-bf79-4576-ae89-9f43a1713e5c","Type":"ContainerStarted","Data":"be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029"}
Oct 30 21:13:46.119895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.119726    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" event={"ID":"7b8f2f70-bf79-4576-ae89-9f43a1713e5c","Type":"ContainerStarted","Data":"2f03ecf27d543e26840304ad81a843d76a4a8eaba4717b812c744e8179919f9d"}
Oct 30 21:13:46.166220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.166158    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podStartSLOduration=1.166141654 podStartE2EDuration="1.166141654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:46.162948185 +0000 UTC m=+808.937348124" watchObservedRunningTime="2026-10-30 21:13:46.166141654 +0000 UTC m=+808.940541594"
Oct 30 21:13:46.610187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.610149    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"]
Oct 30 21:13:46.615259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.615227    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.630946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.630921    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-56ac74\"/\"default-dockercfg-rhfqr\""
Oct 30 21:13:46.631216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.631201    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-56ac74\"/\"distinct-w1-56ac74-kserve-self-signed-certs\""
Oct 30 21:13:46.631658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.631638    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"]
Oct 30 21:13:46.647864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.642966    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-56ac74\"/\"openshift-service-ca.crt\""
Oct 30 21:13:46.647864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.643024    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-56ac74\"/\"kube-root-ca.crt\""
Oct 30 21:13:46.695854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.695529    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:46.719058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.719030    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:46.744537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.744500    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.744723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.744560    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.744723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.744670    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.744876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.744749    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.744876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.744815    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z496m\" (UniqueName: \"kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.745024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.745011    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.845934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.845893    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.845955    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z496m\" (UniqueName: \"kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846121    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846196    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846268    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846311    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846366    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846509    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.846605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.846552    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.848634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.848615    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.848846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.848811    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.868477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.868409    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z496m\" (UniqueName: \"kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m\") pod \"distinct-w1-56ac74-kserve-698db5b784-4kwh6\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") " pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:46.894855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.893952    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:46.934599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:46.934313    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:47.364951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:47.364926    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"]
Oct 30 21:13:47.366495 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:13:47.366465    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb562bdeb_bf87_4c6f_b22e_34792cce8cc8.slice/crio-53b3d32958d1c32ce7cb548855d486d627b1ac0e3f57805d383bb70b04cef758 WatchSource:0}: Error finding container 53b3d32958d1c32ce7cb548855d486d627b1ac0e3f57805d383bb70b04cef758: Status 404 returned error can't find the container with id 53b3d32958d1c32ce7cb548855d486d627b1ac0e3f57805d383bb70b04cef758
Oct 30 21:13:47.769064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:47.768977    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:13:47.769630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:47.769597    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 21:13:48.141704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.141669    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" event={"ID":"b562bdeb-bf87-4c6f-b22e-34792cce8cc8","Type":"ContainerStarted","Data":"e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62"}
Oct 30 21:13:48.141952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.141723    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" event={"ID":"b562bdeb-bf87-4c6f-b22e-34792cce8cc8","Type":"ContainerStarted","Data":"53b3d32958d1c32ce7cb548855d486d627b1ac0e3f57805d383bb70b04cef758"}
Oct 30 21:13:48.190865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.188845    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podStartSLOduration=2.188810728 podStartE2EDuration="2.188810728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:48.187002178 +0000 UTC m=+810.961402119" watchObservedRunningTime="2026-10-30 21:13:48.188810728 +0000 UTC m=+810.963210669"
Oct 30 21:13:48.696084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.696034    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:48.726582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.726546    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:13:48.726897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.726859    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 21:13:48.936856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:48.935873    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:49.718904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:49.718863    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:13:49.719219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:49.718928    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 21:13:50.431318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:50.431267    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:50.718810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:50.718720    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:13:50.718810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:50.718780    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 21:13:51.719015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:51.718973    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:13:51.719470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:51.719059    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 21:13:52.176295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:52.176228    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:52.718501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:52.718464    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:13:52.718693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:52.718539    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 21:13:53.723479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:53.723447    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:53.723931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:53.723725    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:53.724728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:53.724706    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:13:53.732014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:53.731985    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:55.211028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:55.210977    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:56.575286 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:13:56.575251    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:13:56.619971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:56.619914    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:56.936053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:56.935957    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:56.948694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:56.948663    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:57.207393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:57.207299    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:13:58.113591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:58.113559    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:58.448123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:58.448036    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:59.834856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:13:59.834736    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:01.325954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:01.325923    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:02.909584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:02.909549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:04.432477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:04.432446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:04.878782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:04.878742    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:05.888063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:05.888031    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:06.478275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:06.478240    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:07.516920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:07.516892    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:08.450207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:08.450177    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:09.116101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:09.116065    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:10.621201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:10.621167    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:12.071974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:12.071937    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:13.522883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:13.522845    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:15.524447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.524405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:15.615519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.615458    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"]
Oct 30 21:14:15.624814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.624781    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.656882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.656855    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"]
Oct 30 21:14:15.672331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.671839    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.673891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.673814    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"]
Oct 30 21:14:15.674125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.674106    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"]
Oct 30 21:14:15.676394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.676366    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.676660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.676640    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.677091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.677070    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.677272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.677255    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.677448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.677432    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.677600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.677586    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.677772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.677755    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.678024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.678007    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.678198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.678182    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m72z\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.678425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.678408    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.678605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.678588    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.678777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.678762    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.678975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.678960    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.679142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.679128    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.679333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.679311    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.679521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.679507    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4gd4\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.679664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.679649    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.679801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.679788    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781195    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4gd4\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781233    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.781276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781257    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781378    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781438    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781467    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.781570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781489    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.781570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781521    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781553    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.781904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781701    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.781904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781752    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.781904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781788    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m72z\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.781904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781882    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781919    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781929    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781954    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.781979    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782008    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782011    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.782271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782121    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.782651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782324    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.782651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782380    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782640    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782722    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.782812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782798    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.782969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.782901    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.783105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.783082    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.783166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.783111    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.784201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.784176    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.784381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.784356    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.784544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.784527    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.784642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.784627    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.805358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.805330    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.805816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.805794    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4gd4\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.805932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.805794    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.806706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.806683    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m72z\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:15.993852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.993789    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:15.998432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:15.998419    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:16.327144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.327119    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"]
Oct 30 21:14:16.329593 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:14:16.329572    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbe5a2b_bed0_4b5d_b46b_f0a0fced053b.slice/crio-acf1185feeed2616a4d7e0ff48addaac84256ca604241e4f8be5b2df4548997e WatchSource:0}: Error finding container acf1185feeed2616a4d7e0ff48addaac84256ca604241e4f8be5b2df4548997e: Status 404 returned error can't find the container with id acf1185feeed2616a4d7e0ff48addaac84256ca604241e4f8be5b2df4548997e
Oct 30 21:14:16.330500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.330480    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"]
Oct 30 21:14:16.331775 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:14:16.331754    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4b1567_6039_4afc_8d7d_72ff11a0fa69.slice/crio-a22166f0de9cc9e221d26b237a2761658808c2721893a5bf2b446389a8cfec7f WatchSource:0}: Error finding container a22166f0de9cc9e221d26b237a2761658808c2721893a5bf2b446389a8cfec7f: Status 404 returned error can't find the container with id a22166f0de9cc9e221d26b237a2761658808c2721893a5bf2b446389a8cfec7f
Oct 30 21:14:16.332552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.332512    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:16.332634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.332605    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:16.332688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.332648    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:16.334946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.334912    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:16.335050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.334991    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:16.335050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.335036    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:16.503366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.503322    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" event={"ID":"9f4b1567-6039-4afc-8d7d-72ff11a0fa69","Type":"ContainerStarted","Data":"316cc882c8cb7e6feee86131cf91a5fdcf401fda7325a05ca991dc2d02eeb4ee"}
Oct 30 21:14:16.503556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.503409    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" event={"ID":"9f4b1567-6039-4afc-8d7d-72ff11a0fa69","Type":"ContainerStarted","Data":"a22166f0de9cc9e221d26b237a2761658808c2721893a5bf2b446389a8cfec7f"}
Oct 30 21:14:16.507333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.506859    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" event={"ID":"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b","Type":"ContainerStarted","Data":"a7e0cf91ce49a66f960d92301c9b62ff0d3e6f7781a5e3cdb08e44393c1d0f69"}
Oct 30 21:14:16.507333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.506946    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" event={"ID":"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b","Type":"ContainerStarted","Data":"acf1185feeed2616a4d7e0ff48addaac84256ca604241e4f8be5b2df4548997e"}
Oct 30 21:14:16.671570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.671436    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podStartSLOduration=1.6714204929999998 podStartE2EDuration="1.671420493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:16.668753869 +0000 UTC m=+839.443153832" watchObservedRunningTime="2026-10-30 21:14:16.671420493 +0000 UTC m=+839.445820432"
Oct 30 21:14:16.672596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.672560    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podStartSLOduration=1.672547432 podStartE2EDuration="1.672547432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:16.591986143 +0000 UTC m=+839.366386083" watchObservedRunningTime="2026-10-30 21:14:16.672547432 +0000 UTC m=+839.446947375"
Oct 30 21:14:16.905699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.905663    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"]
Oct 30 21:14:16.909794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.909777    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:16.926417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.926098    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-839163\"/\"kube-root-ca.crt\""
Oct 30 21:14:16.931325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.931289    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-839163\"/\"distinct-2-w4-839163-kserve-self-signed-certs\""
Oct 30 21:14:16.931450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.931325    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-839163\"/\"default-dockercfg-xgml5\""
Oct 30 21:14:16.932114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.932088    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"]
Oct 30 21:14:16.965517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.965487    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:16.994616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.994580    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:16.998762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.998732    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:16.999393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.999368    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:16.999453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.999416    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:16.999592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.999577    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:16.999628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.999614    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfx6n\" (UniqueName: \"kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:16.999777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.999749    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:16.999930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:16.999916    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.028855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.026181    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-839163\"/\"openshift-service-ca.crt\""
Oct 30 21:14:17.122856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.122742    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.122856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.122811    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.123119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.122976    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.123119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.123034    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfx6n\" (UniqueName: \"kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.123119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.123086    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.123272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.123202    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.124859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.124392    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.124859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.124702    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.125049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.124970    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.144853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.135758    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.170873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.168943    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.170873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.169612    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfx6n\" (UniqueName: \"kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n\") pod \"distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") " pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.236969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.234657    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:17.405857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.393641    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:17.909752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:17.909720    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"]
Oct 30 21:14:17.944540 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:14:17.944091    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc188c359_c3b2_4ba5_aef9_57044df0bb21.slice/crio-307106f15a1ece397d46bc48762ce862044db7b61ebfbf2b342b3471e662568c WatchSource:0}: Error finding container 307106f15a1ece397d46bc48762ce862044db7b61ebfbf2b342b3471e662568c: Status 404 returned error can't find the container with id 307106f15a1ece397d46bc48762ce862044db7b61ebfbf2b342b3471e662568c
Oct 30 21:14:18.009384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.003286    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:14:18.009384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.003344    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:14:18.021638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.021030    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:14:18.021638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.021116    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:14:18.660020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.659985    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" event={"ID":"c188c359-c3b2-4ba5-aef9-57044df0bb21","Type":"ContainerStarted","Data":"6164741282f6ffc1f8cce2ffbd2d35daf1308ccb26efcbdcbdf9b5a4f1bf334c"}
Oct 30 21:14:18.660256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.660035    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" event={"ID":"c188c359-c3b2-4ba5-aef9-57044df0bb21","Type":"ContainerStarted","Data":"307106f15a1ece397d46bc48762ce862044db7b61ebfbf2b342b3471e662568c"}
Oct 30 21:14:18.722588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:18.722522    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podStartSLOduration=2.722506187 podStartE2EDuration="2.722506187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:18.71917905 +0000 UTC m=+841.493578990" watchObservedRunningTime="2026-10-30 21:14:18.722506187 +0000 UTC m=+841.496906142"
Oct 30 21:14:19.005930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.003531    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:14:19.005930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.003588    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:14:19.005930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.004013    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:14:19.005930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.004048    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:14:19.239856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.236932    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:19.264959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.260669    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:19.997855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.997259    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:14:19.997855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:19.997325    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:14:20.012856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:20.002235    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:14:20.012856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:20.002295    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:14:20.997857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:20.997222    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:14:20.997857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:20.997285    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:14:21.021636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:21.021597    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:14:21.022182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:21.021649    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:14:21.123709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:21.123672    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.994853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:21.994228    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:14:21.994853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:21.994290    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:14:22.002467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.002188    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:14:22.002467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.002244    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:14:22.960307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.960273    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:22.996862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.996228    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:14:22.996862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.996307    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:14:23.000851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.998770    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:14:23.000851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:22.998824    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:14:23.999748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:23.999708    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:24.000310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:24.000171    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:24.000942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:24.000920    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:14:24.512038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:24.512001    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:24.512651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:24.512593    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:24.514477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:24.514458    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:14:24.731372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:24.731333    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:26.132383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:26.132355    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:26.573247 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:14:26.573215    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:14:27.236005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:27.235970    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:27.248568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:27.248545    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:27.694788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:27.694751    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:27.828815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:27.828786    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:14:29.231027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:29.230942    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:30.578413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.578381    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"]
Oct 30 21:14:30.582904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.582877    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.599931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.599905    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"]
Oct 30 21:14:30.604114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.604089    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.613560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.613532    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"]
Oct 30 21:14:30.631033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.631001    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"]
Oct 30 21:14:30.714627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714595    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.714627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714627    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.714865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714744    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.714865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714771    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.714865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714791    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.714993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714901    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.714993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.714958    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhf48\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.715059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715026    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.715092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715067    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.715135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715119    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.715170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715142    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.715212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715196    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.715245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715214    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.715302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715288    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.715336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715310    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.715374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715354    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqzm4\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.715407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715391    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.715440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.715421    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.804166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.793071    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818059    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818131    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818160    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818207    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818237    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818300    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818327    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818358    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqzm4\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818421    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818460    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818559    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818592    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818673    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818689    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818707    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818750    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818767    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhf48\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.818804    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.826849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.819331    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.830033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.829967    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.830462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.830440    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.831128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.831090    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.831519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.831498    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.831918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.831899    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.832289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.832270    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.833095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.833075    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.833683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.833432    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.834459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.834415    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.838474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.837803    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.840907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.840884    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.847854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.843818    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.851865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.849313    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.853764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.853661    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.863506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.863287    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.863506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.863430    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqzm4\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:30.863798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.863757    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhf48\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48\") pod \"e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") " pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.897387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.897358    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:30.921873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:30.921819    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:31.230101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.230071    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"]
Oct 30 21:14:31.231774 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:14:31.231747    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234f9898_40ad_4244_879d_ddc6cdb27830.slice/crio-8d1b882e215c292bbf62c7ae7972db4c4019e8d4be0a7096c5b259feeb69d7fd WatchSource:0}: Error finding container 8d1b882e215c292bbf62c7ae7972db4c4019e8d4be0a7096c5b259feeb69d7fd: Status 404 returned error can't find the container with id 8d1b882e215c292bbf62c7ae7972db4c4019e8d4be0a7096c5b259feeb69d7fd
Oct 30 21:14:31.234237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.234203    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:31.234303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.234269    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:31.234340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.234300    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:31.267959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.267934    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"]
Oct 30 21:14:31.269026 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:14:31.269001    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435a34bd_c5d8_4d7d_bb65_a4ccd612f791.slice/crio-e17ad781ba21cce6bca5f4e5c6bd6486dc7faf5ffbe3fa4ac168c333da96700e WatchSource:0}: Error finding container e17ad781ba21cce6bca5f4e5c6bd6486dc7faf5ffbe3fa4ac168c333da96700e: Status 404 returned error can't find the container with id e17ad781ba21cce6bca5f4e5c6bd6486dc7faf5ffbe3fa4ac168c333da96700e
Oct 30 21:14:31.271583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.271547    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:31.271691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.271615    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:31.271691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.271646    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:31.845435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.845402    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" event={"ID":"435a34bd-c5d8-4d7d-bb65-a4ccd612f791","Type":"ContainerStarted","Data":"fc2a397f78b8edd7a85e323b79f355f48aecfa69ce5c1c5e583e8630cd04101c"}
Oct 30 21:14:31.845917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.845464    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" event={"ID":"435a34bd-c5d8-4d7d-bb65-a4ccd612f791","Type":"ContainerStarted","Data":"e17ad781ba21cce6bca5f4e5c6bd6486dc7faf5ffbe3fa4ac168c333da96700e"}
Oct 30 21:14:31.847510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.847473    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" event={"ID":"234f9898-40ad-4244-879d-ddc6cdb27830","Type":"ContainerStarted","Data":"9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9"}
Oct 30 21:14:31.847625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.847532    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" event={"ID":"234f9898-40ad-4244-879d-ddc6cdb27830","Type":"ContainerStarted","Data":"8d1b882e215c292bbf62c7ae7972db4c4019e8d4be0a7096c5b259feeb69d7fd"}
Oct 30 21:14:31.897492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.897440    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:31.922277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.922256    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:31.928196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.928153    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podStartSLOduration=1.9281412329999998 podStartE2EDuration="1.928141233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:31.928085206 +0000 UTC m=+854.702485146" watchObservedRunningTime="2026-10-30 21:14:31.928141233 +0000 UTC m=+854.702541172"
Oct 30 21:14:31.994608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:31.994564    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podStartSLOduration=1.994551092 podStartE2EDuration="1.994551092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:31.9890815 +0000 UTC m=+854.763481438" watchObservedRunningTime="2026-10-30 21:14:31.994551092 +0000 UTC m=+854.768951031"
Oct 30 21:14:32.563420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:32.563384    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.915857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:32.911969    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:14:32.915857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:32.912053    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:14:32.923579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:32.923547    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:14:32.923732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:32.923598    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:14:33.898854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:33.898191    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:14:33.898854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:33.898249    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:14:33.930611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:33.930486    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:14:33.930611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:33.930546    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:14:34.329559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:34.329532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:34.909309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:34.909267    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:14:34.909660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:34.909629    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:14:34.923136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:34.923101    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:14:34.923281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:34.923154    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:14:35.898867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:35.898271    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:14:35.898867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:35.898333    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:14:35.922919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:35.922883    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:14:35.923139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:35.923120    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:14:36.121466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:36.121428    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:36.898857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:36.898402    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:14:36.898857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:36.898460    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:14:36.922458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:36.922411    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:14:36.922660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:36.922478    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:14:37.759578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:37.759547    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:37.899176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:37.899145    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:14:37.899648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:37.899624    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:14:37.922428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:37.922397    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:14:37.922567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:37.922452    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:14:38.037939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.037800    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:38.930088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.930055    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:38.931458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.931285    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:38.935955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.935929    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:14:38.983508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.983471    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:38.985275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.985251    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:38.987574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:38.987520    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:14:39.406078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:39.406040    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:41.785308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:41.785271    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:42.745107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:42.745027    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:14:42.745455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:42.745423    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:42.946110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:42.946059    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:43.232855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:43.223967    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:43.378856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:43.378257    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:14:43.438856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:43.436931    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:43.827857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:43.826931    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:44.620169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:44.620135    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:45.593113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.593078    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"]
Oct 30 21:14:45.598044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.598019    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.630287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.630247    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"]
Oct 30 21:14:45.765778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.765744    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.765959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.765798    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.765959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.765939    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.766047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.765980    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.766083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.766058    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.766083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.766076    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.766181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.766164    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.766244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.766232    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.766370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.766357    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrhtt\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867323    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrhtt\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867376    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867409    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867442    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867465    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867498    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867515    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867574    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.867705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.867659    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.868127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.868108    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.868497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.868467    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.868633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.868503    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.868633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.868599    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.868756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.868633    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.870521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.870486    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.870625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.870609    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.888132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.888108    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.888460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.888441    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrhtt\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt\") pod \"e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") " pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:45.919798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:45.919772    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:46.102316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:46.102279    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:46.283479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:46.283359    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"]
Oct 30 21:14:46.284534 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:14:46.284505    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8958b7e_cf5b_44b2_ae11_cb9a4841fb24.slice/crio-97244d63155d833c9034875beaf13a8d76af97616472aca4c46214374d0b5663 WatchSource:0}: Error finding container 97244d63155d833c9034875beaf13a8d76af97616472aca4c46214374d0b5663: Status 404 returned error can't find the container with id 97244d63155d833c9034875beaf13a8d76af97616472aca4c46214374d0b5663
Oct 30 21:14:46.286798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:46.286761    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:46.286941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:46.286879    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:46.287003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:46.286939    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:47.144199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.144160    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" event={"ID":"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24","Type":"ContainerStarted","Data":"32089c143348fb668a44ac06925bc27ff2b19596086aa32c98855faac7c053f6"}
Oct 30 21:14:47.144708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.144215    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" event={"ID":"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24","Type":"ContainerStarted","Data":"97244d63155d833c9034875beaf13a8d76af97616472aca4c46214374d0b5663"}
Oct 30 21:14:47.227069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.227017    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podStartSLOduration=2.227003715 podStartE2EDuration="2.227003715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:47.214636342 +0000 UTC m=+869.989036280" watchObservedRunningTime="2026-10-30 21:14:47.227003715 +0000 UTC m=+870.001403654"
Oct 30 21:14:47.655473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.655439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:47.920568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.920476    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:47.922079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.922052    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:14:47.922210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:47.922106    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:14:48.920775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:48.920734    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:14:48.921200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:48.920789    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:14:49.344984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:49.344949    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:49.920637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:49.920600    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:14:49.920890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:49.920668    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:14:50.921051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:50.921020    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:14:50.921447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:50.921082    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:14:50.937340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:50.937301    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:51.920404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:51.920370    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:14:51.920595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:51.920434    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:14:52.536347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:52.536313    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:52.920797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:52.920759    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:14:52.921025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:52.920854    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 21:14:53.223290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:53.223197    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:53.976110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:53.976076    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:53.976696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:53.976669    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:53.978081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:53.978060    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:14:54.341543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:54.341514    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:55.932888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:55.932844    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:56.574585 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:14:56.574549    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:14:57.394315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:57.394282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:58.934512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:14:58.934469    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:00.553847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:00.553801    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:02.407906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:02.406771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:04.000684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:04.000649    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:05.067685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:05.067651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:05.195172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:05.195134    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:05.538973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:05.538869    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:07.043172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:07.043136    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:08.778705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:08.778669    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:10.501038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:10.500980    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:10.833960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:10.833910    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:10.885980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:10.885941    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:10.886255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:10.886221    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:11.167912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.153986    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:11.204970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.204921    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:11.316856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.309276    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:11.316856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.309342    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:11.389897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.381342    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:11.479916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.473064    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:11.876947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.874160    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:15:11.876947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:11.875728    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:15:12.056536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.056492    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:15:12.195048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.154952    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:12.196509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.195339    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:15:12.208065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.206931    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:12.208065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.207789    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:15:12.225476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.223920    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:12.637858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.612326    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:15:12.637858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.612914    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:15:12.925965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:12.919361    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:13.135961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.135899    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:15:13.207070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.206970    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:13.207349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.207318    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:13.215956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.215904    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:13.216696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.216491    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:15:13.217192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.217081    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:15:13.219857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.219452    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:15:13.260080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.259676    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:13.693907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:13.636554    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:14.011488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.011398    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:15:14.132851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.118902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:14.186851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.182112    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:14.195556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.195523    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:15:14.222024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.221982    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:14.222636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.222605    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:15:14.304091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.304011    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:15:14.352854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.350807    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:15:14.408999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.408106    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:15:14.409193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:14.409115    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:15:15.671764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.671724    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"]
Oct 30 21:15:15.676099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.676074    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.737599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.737562    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.737799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.737678    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfr9t\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.737799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.737714    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.737799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.737793    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.738008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.737822    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.738008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.737954    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.738117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.738058    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.738168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.738155    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.738218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.738202    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.741597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.741574    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"]
Oct 30 21:15:15.839065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839025    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839106    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839136    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839209    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839234    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfr9t\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839262    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839331    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839384    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839413    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839597    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839648    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839693    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839792    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.839993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.839959    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.841890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.841865    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.842147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.842128    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.864014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.863983    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.864135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.864052    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfr9t\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:15.993173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:15.993091    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:16.000396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.000357    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:16.446510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.446477    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"]
Oct 30 21:15:16.447150 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:15:16.447123    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8e0a11_ee33_4e51_826d_8636f62d79b1.slice/crio-5015f1feaa1f309f124ef4318873a65d995b97e5db7d7fdb1bae892dacf141f5 WatchSource:0}: Error finding container 5015f1feaa1f309f124ef4318873a65d995b97e5db7d7fdb1bae892dacf141f5: Status 404 returned error can't find the container with id 5015f1feaa1f309f124ef4318873a65d995b97e5db7d7fdb1bae892dacf141f5
Oct 30 21:15:16.450308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.450275    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:16.450398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.450348    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:16.450398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.450385    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:16.571026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.570980    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"]
Oct 30 21:15:16.596351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.596314    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.612296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.612267    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"]
Oct 30 21:15:16.626075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.625292    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"]
Oct 30 21:15:16.626075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.625400    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.653322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.653292    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"]
Oct 30 21:15:16.654690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.654663    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.654944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.654923    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655089    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655295    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmnpb\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655521    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655588    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655611    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655755    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.655846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.655788    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.756960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.756875    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vws7\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.756960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.756915    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757004    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757049    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757079    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757123    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757264    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757321    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757386    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757423    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757473    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757495    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757518    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmnpb\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.757885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757547    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757579    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.757885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757647    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.758200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.757886    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.758200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.758028    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.758200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.758037    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.758200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.758084    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.758200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.758125    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.758579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.758554    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.759007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.758944    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.760063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.760042    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.760367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.760351    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.785988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.785960    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.786792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.786767    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmnpb\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.859576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859548    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.859759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859589    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.859759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859609    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.859759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859677    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.859959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859820    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vws7\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.859959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859891    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.859959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859944    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.860103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.859987    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.860103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.860083    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.860191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.860106    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.860443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.860418    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.860539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.860491    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.860628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.860602    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.861045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.860918    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.862645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.862617    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.862789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.862769    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.883593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.883554    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vws7\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.884727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.884696    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token\") pod \"e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") " pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:16.945382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.945347    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:16.965551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:16.965523    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:17.303493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.302173    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"]
Oct 30 21:15:17.309367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.309103    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"]
Oct 30 21:15:17.318585 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:15:17.318555    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08643223_5f4c_44cd_be86_9c9c7aee0a84.slice/crio-817e41d128027e22bcdb23b6c93644b2479454aaf57f109c2240f641fb3cac29 WatchSource:0}: Error finding container 817e41d128027e22bcdb23b6c93644b2479454aaf57f109c2240f641fb3cac29: Status 404 returned error can't find the container with id 817e41d128027e22bcdb23b6c93644b2479454aaf57f109c2240f641fb3cac29
Oct 30 21:15:17.321170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.321132    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:17.321265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.321228    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:17.321327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.321276    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:17.321539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.321511    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" event={"ID":"0b8e0a11-ee33-4e51-826d-8636f62d79b1","Type":"ContainerStarted","Data":"e90d1d052b10ec3919ab233bb504fd14afadfa3818d6f46c920087e9d2f29b3f"}
Oct 30 21:15:17.321638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.321568    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" event={"ID":"0b8e0a11-ee33-4e51-826d-8636f62d79b1","Type":"ContainerStarted","Data":"5015f1feaa1f309f124ef4318873a65d995b97e5db7d7fdb1bae892dacf141f5"}
Oct 30 21:15:17.322307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.322279    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:17.322393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.322348    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:17.322437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.322393    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:17.410637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.410580    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podStartSLOduration=2.410558757 podStartE2EDuration="2.410558757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:17.403253058 +0000 UTC m=+900.177652997" watchObservedRunningTime="2026-10-30 21:15:17.410558757 +0000 UTC m=+900.184958698"
Oct 30 21:15:17.546959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.543643    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:15:17.546959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.546358    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:15:17.997041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.997003    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:17.998873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.998819    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:17.999151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:17.999102    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:18.272912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.269246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:18.373853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.367911    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" event={"ID":"08643223-5f4c-44cd-be86-9c9c7aee0a84","Type":"ContainerStarted","Data":"11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82"}
Oct 30 21:15:18.373853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.367990    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" event={"ID":"08643223-5f4c-44cd-be86-9c9c7aee0a84","Type":"ContainerStarted","Data":"817e41d128027e22bcdb23b6c93644b2479454aaf57f109c2240f641fb3cac29"}
Oct 30 21:15:18.400857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.387307    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" event={"ID":"b1baa751-388b-42f2-829c-7dfc01f41627","Type":"ContainerStarted","Data":"cb6ac08dd4dda2ffad35159ec641ee6626585dfb93cd248185998a4bdf34a7d8"}
Oct 30 21:15:18.400857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.387366    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" event={"ID":"b1baa751-388b-42f2-829c-7dfc01f41627","Type":"ContainerStarted","Data":"18987d188c84c2db947f8a5ec3cf7a185f9578fd3035a15f828f6df0a3800751"}
Oct 30 21:15:18.465065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.465002    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podStartSLOduration=2.464975527 podStartE2EDuration="2.464975527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:18.448327137 +0000 UTC m=+901.222727077" watchObservedRunningTime="2026-10-30 21:15:18.464975527 +0000 UTC m=+901.239375467"
Oct 30 21:15:18.767748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.764429    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podStartSLOduration=2.764408318 podStartE2EDuration="2.764408318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:18.517771361 +0000 UTC m=+901.292171301" watchObservedRunningTime="2026-10-30 21:15:18.764408318 +0000 UTC m=+901.538808258"
Oct 30 21:15:18.951405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.950743    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:18.951405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.950918    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:18.951405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.950968    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:18.966393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.966369    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:18.967244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.967220    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:18.967345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.967270    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:18.998613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.998579    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:18.999280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:18.999247    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:19.948199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:19.948160    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:19.950768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:19.948399    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:19.966625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:19.966593    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:19.966759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:19.966640    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:19.997250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:19.997211    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:19.997537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:19.997507    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:20.798544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.792598    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:20.955543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.951588    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:20.955543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.951647    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:20.968334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.968209    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:20.968334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.968271    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:20.997082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.996360    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:20.997082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:20.996418    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:21.962649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:21.962608    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:21.963124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:21.962703    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:21.967252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:21.967221    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:21.967497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:21.967467    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:22.062263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:22.042987    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:22.062263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:22.043102    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:22.952613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:22.952572    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:22.953003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:22.952971    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:22.965920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:22.965894    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:22.966341 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:22.966319    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:23.039586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.033002    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:23.039586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.033089    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:23.107890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.107510    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:23.946348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.946307    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:23.946674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.946373    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:23.966709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.966669    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:23.967133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:23.966737    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:24.015730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.015686    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:15:24.016119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.016087    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 21:15:24.946230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.946191    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:15:24.946465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.946261    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 21:15:24.968329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.968288    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:15:24.968881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.968357    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 21:15:24.998846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.998798    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:24.999137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:24.999117    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:25.002240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.002218    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:15:25.024457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.024421    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:25.950301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.950271    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:25.950705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.950687    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:25.951412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.951393    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:15:25.969933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.969912    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:25.970259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.970187    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:25.970841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:25.970807    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:15:26.574122 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:15:26.574083    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:15:26.659914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:26.659880    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:28.227227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:28.227193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:29.820444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:29.820408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:30.633301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.633266    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"]
Oct 30 21:15:30.637747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.637723    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693436    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8587w\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693491    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693561    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693679    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693746    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693870    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.693979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.693946    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.694721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.694005    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.694721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.694057    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.702096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.702057    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"]
Oct 30 21:15:30.795084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795028    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795119    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795151    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795181    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795211    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795263    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8587w\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795285    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795343    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795442    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795590    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795641    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795799    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795879    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.795931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.795883    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.797720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.797697    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.797937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.797917    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.820254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.820223    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.820426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.820293    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8587w\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w\") pod \"e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") " pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:30.955220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:30.955132    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:31.276104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.276068    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"]
Oct 30 21:15:31.276892 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:15:31.276816    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92e1b1c_a803_4186_b935_4f0cc43b3d85.slice/crio-24d3761846b26acebd296e407669fcf5912af4a0a7ee861ee4fd4df9a0ff1cd1 WatchSource:0}: Error finding container 24d3761846b26acebd296e407669fcf5912af4a0a7ee861ee4fd4df9a0ff1cd1: Status 404 returned error can't find the container with id 24d3761846b26acebd296e407669fcf5912af4a0a7ee861ee4fd4df9a0ff1cd1
Oct 30 21:15:31.280487 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.280444    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:31.280593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.280538    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:31.280593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.280584    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:31.504369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.504335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:31.712851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.712798    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" event={"ID":"c92e1b1c-a803-4186-b935-4f0cc43b3d85","Type":"ContainerStarted","Data":"d8ea42a4defa9da41a3afcb4935765d7c9760f045f15cdad1cdb6748665a8829"}
Oct 30 21:15:31.713037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.712878    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" event={"ID":"c92e1b1c-a803-4186-b935-4f0cc43b3d85","Type":"ContainerStarted","Data":"24d3761846b26acebd296e407669fcf5912af4a0a7ee861ee4fd4df9a0ff1cd1"}
Oct 30 21:15:31.772954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.772890    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podStartSLOduration=1.772867354 podStartE2EDuration="1.772867354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:31.770227241 +0000 UTC m=+914.544627179" watchObservedRunningTime="2026-10-30 21:15:31.772867354 +0000 UTC m=+914.547267292"
Oct 30 21:15:31.955428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:31.955398    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:32.957056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:32.957018    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:15:32.957465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:32.957067    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:15:33.053285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:33.053237    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:33.956039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:33.956006    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:15:33.956232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:33.956066    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:15:34.893855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:34.893705    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:34.956853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:34.956252    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:15:34.956853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:34.956363    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:15:35.956386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:35.956347    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:15:35.956804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:35.956414    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:15:36.658979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:36.658947    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:36.956444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:36.956354    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:15:36.956444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:36.956411    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:15:37.955910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:37.955871    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:15:37.956094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:37.955938    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:15:38.382020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:38.381984    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:39.101152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:39.101116    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:39.101941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:39.101919    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:39.105403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:39.105379    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:15:39.546059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:39.545970    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:40.230975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:40.230930    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:40.245285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:40.245237    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:40.492910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:40.486941    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:40.715855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:40.711391    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:15:41.191241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:41.189938    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:41.744033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:41.743982    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:41.767859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:41.755771    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:15:42.142856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.106656    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:15:42.234890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.234722    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:42.274204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.240972    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:42.298884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.241013    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:15:42.299755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.241035    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:15:42.324858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.300189    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:15:42.351342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.351273    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:42.785860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:42.769558    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.148010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.147954    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.150945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.149054    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:15:43.315078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.315016    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:15:43.320560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.317303    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:15:43.320560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.317615    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:15:43.320560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.318678    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:15:43.320560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.318736    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:15:43.454096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:43.453964    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:15:44.161325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.161272    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.293173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.293126    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.380858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.372923    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct" podUID="569e7b40-df79-438b-a350-7f4cf8202c9c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.393858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.393173    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:15:44.401872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.394141    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.413348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.400944    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.478852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.462462    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:15:44.485867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.472881    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.501857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:44.475868    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:45.198350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.197179    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.301984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.301930    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.403996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.403949    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.404668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.404632    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": context deadline exceeded"
Oct 30 21:15:45.421983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.415361    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:15:45.453169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.446048    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:15:45.492113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.484935    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:45.492113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.485016    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.502994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:45.502942    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.279907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.268345    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.345634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.311982    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.345634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.320976    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"]
Oct 30 21:15:46.468865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.435611    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.481952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.463273    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.481952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.463317    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.504083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.504029    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.547931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.536120    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"]
Oct 30 21:15:46.573898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.546996    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.576977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.572863    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.577566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.577537    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.577866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.577823    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.578080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.578063    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.578324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.578307    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.578498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.578481    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2r7l\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.581968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.581941    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.582294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.582274    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.583392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.583370    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.704859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.689815    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.713965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.713921    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.714433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.714406    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.714581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.714561    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.714753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.714735    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.714951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.714933    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.715192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.715174    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.715316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.715301    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2r7l\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.715455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.715437    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.716262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.716236    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.716422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.704228    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.741860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.720485    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.742505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.725899    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.742666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.726132    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.754746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.754712    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.817922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.802607    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2r7l\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.822857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.819526    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:46.847861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:46.834268    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy\") pod \"e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") " pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:47.024175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.024130    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:47.285938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.281319    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:15:47.346852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.323869    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:15:47.576917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.576554    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:15:47.673891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.647385    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:15:47.698479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.698427    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:15:47.728031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:47.701342    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.021911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.011211    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.333915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.307265    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.369901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.369857    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.579915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.562313    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.579915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.562722    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.589984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.585223    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.705984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:48.705930    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:49.034062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.034016    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:49.043274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.034137    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:15:49.160511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.158643    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:15:49.160511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.159072    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:15:49.160511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.159799    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:15:49.160849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.160578    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:15:49.180953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.176773    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:15:49.186599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.183091    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:15:49.346027 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:15:49.345990    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba9d8ef_3105_4591_af9e_9fee8a9c577d.slice/crio-f14a4ab065ca96a8ee29fd05e8df8bec0b62e90dfe64819259d653f999c66a91 WatchSource:0}: Error finding container f14a4ab065ca96a8ee29fd05e8df8bec0b62e90dfe64819259d653f999c66a91: Status 404 returned error can't find the container with id f14a4ab065ca96a8ee29fd05e8df8bec0b62e90dfe64819259d653f999c66a91
Oct 30 21:15:49.349203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.349150    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:49.349325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.349271    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:49.349325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.349315    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:49.351931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.351911    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"]
Oct 30 21:15:49.656557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.656453    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" event={"ID":"2ba9d8ef-3105-4591-af9e-9fee8a9c577d","Type":"ContainerStarted","Data":"3cabc901e600f447922fd6553310a555903ca5562abd445d94e0212d5d24573e"}
Oct 30 21:15:49.656557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.656513    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" event={"ID":"2ba9d8ef-3105-4591-af9e-9fee8a9c577d","Type":"ContainerStarted","Data":"f14a4ab065ca96a8ee29fd05e8df8bec0b62e90dfe64819259d653f999c66a91"}
Oct 30 21:15:49.713447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:49.713380    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podStartSLOduration=4.71251352 podStartE2EDuration="4.71251352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:49.709303092 +0000 UTC m=+932.483703030" watchObservedRunningTime="2026-10-30 21:15:49.71251352 +0000 UTC m=+932.486913488"
Oct 30 21:15:50.025631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:50.025542    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:50.027412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:50.027383    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:50.027543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:50.027438    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:50.237325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:50.237288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:51.025618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:51.025580    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:51.026165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:51.025647    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:52.024211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:52.024174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:52.025216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:52.025193    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:52.025289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:52.025241    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:53.025912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:53.025859    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:53.026412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:53.025937    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:53.695381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:53.695329    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:54.025621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:54.025533    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:54.025621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:54.025589    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:55.025655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:55.025614    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:55.026201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:55.025683    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:55.347606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:55.347554    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:56.025104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:56.025066    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:15:56.025286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:56.025139    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 21:15:56.597045 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:15:56.596997    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:15:56.944741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:56.944653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:57.050855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:57.049955    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:57.054439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:57.051721    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:57.059198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:57.059153    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:15:58.437273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:58.437170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:59.996860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:15:59.996292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:01.684378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:01.684331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:02.555656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.555608    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-8ed324"]
Oct 30 21:16:02.560985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.560959    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:02.575320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.575287    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-8ed324"]
Oct 30 21:16:02.611128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.611073    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxp2m\""
Oct 30 21:16:02.699856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.689019    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8j5\" (UniqueName: \"kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5\") pod \"test-curl-7401-8ed324\" (UID: \"2e95ec16-6cfe-4d74-a549-caed307d20c3\") " pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:02.835548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.835514    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8j5\" (UniqueName: \"kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5\") pod \"test-curl-7401-8ed324\" (UID: \"2e95ec16-6cfe-4d74-a549-caed307d20c3\") " pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:02.911957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.911868    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8j5\" (UniqueName: \"kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5\") pod \"test-curl-7401-8ed324\" (UID: \"2e95ec16-6cfe-4d74-a549-caed307d20c3\") " pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:02.945014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:02.941581    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:04.534920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:04.519923    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:04.657788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:04.657751    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-8ed324"]
Oct 30 21:16:04.782896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:04.755944    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:04.873877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:04.865165    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:16:04.948998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:04.948942    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:05.143593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.143548    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:05.177485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.177436    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:05.217132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.217076    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-8ed324" event={"ID":"2e95ec16-6cfe-4d74-a549-caed307d20c3","Type":"ContainerStarted","Data":"b7eed29c688f2717251e4efcee75f7e3caf39b28ae0f5891c34f9da618329b46"}
Oct 30 21:16:05.392263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.392231    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:05.429258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.429153    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:05.432743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.429519    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:05.600857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.577920    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:05.600857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.580102    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:16:05.667874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.617148    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:05.776941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.762026    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:05.776941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.763548    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:16:05.924910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:05.902283    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:06.191282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.191234    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:06.192394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.192369    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:16:06.284155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.284117    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:06.464096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.463985    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:16:06.475864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.465362    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:16:06.561857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.532998    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:06.565323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.562411    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:16:06.733962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.699924    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:06.792293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.792246    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:06.971857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.954495    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:06.986646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:06.986557    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:16:07.200523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:07.193246    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:07.584308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:07.584264    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:07.628901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:07.618301    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:07.708752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:07.708704    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:07.932269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:07.932160    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:08.097328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.097271    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:08.162393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.162354    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:16:08.197855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.197717    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:08.540028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.514932    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Oct 30 21:16:08.596085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.596047    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:08.642872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.620516    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:08.643583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.621506    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:08.713856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.713769    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:08.893442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.893401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:08.916978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:08.916925    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": context deadline exceeded"
Oct 30 21:16:09.074858 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:09.022909    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.418s"
Oct 30 21:16:09.133972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.133940    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:16:09.136482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.136408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:09.314105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.313704    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:09.349866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.322322    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:09.434864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.420930    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:09.434864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.420990    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:09.664123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.664082    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:09.718899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.717963    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:09.777898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.777860    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:09.827192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.827132    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:09.889942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:09.867850    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:10.011167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.011069    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.064032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.063993    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:10.068959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.068912    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.069338 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.069311    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:10.069535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.069506    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.183854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.158663    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:10.267906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.258588    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:10.267906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.258687    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.457642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.457586    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.511800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.481908    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.532191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.494926    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.664676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.664616    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:16:10.748891 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:10.723786    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.365s"
Oct 30 21:16:10.750278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.724010    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:10.750497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.750471    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.750630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.724200    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.907856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.900122    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:10.951425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.945367    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:16:10.959578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.952925    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:10.963852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.960144    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:16:10.978422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.978392    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:16:10.979961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:10.979942    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:16:11.114496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:11.114469    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:16:11.114947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:11.114926    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.182276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:11.182161    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:16:11.380875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:11.379364    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:16:12.181761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.181723    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.322177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.322139    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.470941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.467998    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.603313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.603277    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.688346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.688315    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"]
Oct 30 21:16:12.698102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.698071    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.739432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.739346    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"]
Oct 30 21:16:12.746690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.746661    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-92d5d744-openshift-default-dockercfg-pfrfj\""
Oct 30 21:16:12.757406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.757373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.765908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.765871    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766005    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766093    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766171    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgqd\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766325    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766360    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766507    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766668    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.766783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.766761    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.875991    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876103    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876155    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876265    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876320    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876387    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.876499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876456    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgqd\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876608    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.876642    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.877224    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.882176    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.882806    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.883727    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.903854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.883981    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.926893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.926434    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.959856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.940186    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.959856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.959433    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.975850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.968260    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgqd\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:12.975850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:12.968909    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token\") pod \"e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") " pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:13.043793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.039364    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:13.102853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.094397    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:13.420883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.370711    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:13.603638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.603594    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"]
Oct 30 21:16:13.660485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.658394    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:13.660485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.658490    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:13.660485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.658536    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:13.732790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.732759    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"]
Oct 30 21:16:13.741796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.740248    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.746334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.744135    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"]
Oct 30 21:16:13.759337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.758018    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4b62455c-openshift-default-dockercfg-qjcch\""
Oct 30 21:16:13.788667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.788627    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.788876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.788697    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.788945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.788874    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.788945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.788911    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.789161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.789140    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.789261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.789244    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.789305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.789276    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdcxg\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.789348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.789318    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.789398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.789359    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.818285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.818250    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910020    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910131    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910164    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdcxg\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910202    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910240    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910318    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910360    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910487    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.915855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.910519    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.928947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.927698    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.978312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.972803    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.978312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.973580    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.978312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.974711    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:13.978312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:13.975948    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:14.082294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.080762    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:14.125198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.115397    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:14.125735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.125706    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:14.131806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.127563    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:14.152638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.151088    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdcxg\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg\") pod \"e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") " pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:14.184090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.184004    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"]
Oct 30 21:16:14.215389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.215362    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.248861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.242102    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-dd1e164d-dockercfg-492ps\""
Oct 30 21:16:14.269859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.252238    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"]
Oct 30 21:16:14.269859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.269497    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:14.271817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.271761    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-dd1e164d\""
Oct 30 21:16:14.415370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.415332    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.415661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.415638    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-724vn\" (UniqueName: \"kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn\") pod \"payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.430240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.429975    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:14.532657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.520927    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.532657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.521014    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-724vn\" (UniqueName: \"kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn\") pod \"payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.532657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.522680    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.696422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.670317    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-724vn\" (UniqueName: \"kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn\") pod \"payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.709678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.709088    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:14.716680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.715149    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" event={"ID":"439dd61b-80cd-4a64-b31c-4fd3c2023c6b","Type":"ContainerStarted","Data":"77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f"}
Oct 30 21:16:14.716680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.715243    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" event={"ID":"439dd61b-80cd-4a64-b31c-4fd3c2023c6b","Type":"ContainerStarted","Data":"dc1d669a07337ebdcecba10a2c180620da49ff505c2fb3235ad17d26bf56b596"}
Oct 30 21:16:14.774849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:14.774772    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" podStartSLOduration=2.77475305 podStartE2EDuration="2.77475305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:14.773103935 +0000 UTC m=+957.547503875" watchObservedRunningTime="2026-10-30 21:16:14.77475305 +0000 UTC m=+957.549152990"
Oct 30 21:16:15.045789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.041902    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:15.053392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.053340    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"]
Oct 30 21:16:15.066240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.066177    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:15.077367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.077330    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"]
Oct 30 21:16:15.083033 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:15.083006    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c10a9b9_7c4e_46b9_a4b2_6b7e3a6fcf17.slice/crio-1200b4b447a60de940ff501e4589cd013a9ea91f43c9b1faa24a4ea0bfbff6ba WatchSource:0}: Error finding container 1200b4b447a60de940ff501e4589cd013a9ea91f43c9b1faa24a4ea0bfbff6ba: Status 404 returned error can't find the container with id 1200b4b447a60de940ff501e4589cd013a9ea91f43c9b1faa24a4ea0bfbff6ba
Oct 30 21:16:15.085558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.085526    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:15.085667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.085593    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:15.085667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.085623    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:15.186751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.186714    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:15.724562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.724521    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-8ed324" event={"ID":"2e95ec16-6cfe-4d74-a549-caed307d20c3","Type":"ContainerStarted","Data":"81f70e1ea18edddaa67a0b9403b5fb0cd78701771c41376dfa7e409d3a90d47e"}
Oct 30 21:16:15.726305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.726274    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" event={"ID":"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17","Type":"ContainerStarted","Data":"f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206"}
Oct 30 21:16:15.726451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.726329    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" event={"ID":"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17","Type":"ContainerStarted","Data":"1200b4b447a60de940ff501e4589cd013a9ea91f43c9b1faa24a4ea0bfbff6ba"}
Oct 30 21:16:15.728469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.728432    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9" event={"ID":"befc60ca-1fd8-429d-bc60-bdda7c61bca1","Type":"ContainerStarted","Data":"7bf97fb65417454f09d1febdc16f91445032e86c9234f0b06b49bed89bbb330d"}
Oct 30 21:16:15.728705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.728474    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9" event={"ID":"befc60ca-1fd8-429d-bc60-bdda7c61bca1","Type":"ContainerStarted","Data":"c71032ade55136e77218a52c653c2a66863f77314954e4031ec267f8308709f3"}
Oct 30 21:16:15.728705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.728659    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:15.728705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.728705    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:15.729645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.729626    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:16:15.762622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.762571    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7401-8ed324" podStartSLOduration=3.582142005 podStartE2EDuration="13.762556798s" totalImagesPullingTime="10.180414793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:15.760077753 +0000 UTC m=+958.534477692" watchObservedRunningTime="2026-10-30 21:16:15.762556798 +0000 UTC m=+958.536956736"
Oct 30 21:16:15.796990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.796938    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9" podStartSLOduration=1.796923289 podStartE2EDuration="1.796923289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:15.796255695 +0000 UTC m=+958.570655638" watchObservedRunningTime="2026-10-30 21:16:15.796923289 +0000 UTC m=+958.571323227"
Oct 30 21:16:15.886692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.886659    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"]
Oct 30 21:16:15.891108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.891050    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" podStartSLOduration=2.891029466 podStartE2EDuration="2.891029466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:15.88777887 +0000 UTC m=+958.662178804" watchObservedRunningTime="2026-10-30 21:16:15.891029466 +0000 UTC m=+958.665429406"
Oct 30 21:16:15.892912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.892888    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.905685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.905660    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.905819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.905717    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.905933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.905905    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.906005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.905980    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.906117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.906097    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tqsf\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.906247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.906135    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.906247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.906173    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.906335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.906302    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.906335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.906331    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:15.913337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:15.913307    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"]
Oct 30 21:16:16.008156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.008054    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.008352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.008202    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.008352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.008302    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.008462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.008390    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.008565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.008541    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.033854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.030457    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.033854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.031694    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.033854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.032094    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.044867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.043143    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tqsf\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.044867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.043215    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.044867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.043261    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.044867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.043371    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.044867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.043423    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.044867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.044510    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.084855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.080252    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.088855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.085104    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.147856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.097883    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tqsf\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.147856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.110403    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token\") pod \"e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") " pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.211722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.211689    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:16.406288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.406252    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.436385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.436351    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:16.442215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.442192    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:16.547453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.547421    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"]
Oct 30 21:16:16.548263 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:16.548234    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13226cae_6ab7_44ac_9070_926073140741.slice/crio-0bc5ec03bb1e26f3ecda2147ee2fd6289137f4aea52226e3972a264b88017705 WatchSource:0}: Error finding container 0bc5ec03bb1e26f3ecda2147ee2fd6289137f4aea52226e3972a264b88017705: Status 404 returned error can't find the container with id 0bc5ec03bb1e26f3ecda2147ee2fd6289137f4aea52226e3972a264b88017705
Oct 30 21:16:16.550989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.550951    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:16.551072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.551025    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:16.551072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.551057    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:16.578497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.578459    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.737342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.737289    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" event={"ID":"13226cae-6ab7-44ac-9070-926073140741","Type":"ContainerStarted","Data":"b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab"}
Oct 30 21:16:16.737891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.737362    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" event={"ID":"13226cae-6ab7-44ac-9070-926073140741","Type":"ContainerStarted","Data":"0bc5ec03bb1e26f3ecda2147ee2fd6289137f4aea52226e3972a264b88017705"}
Oct 30 21:16:16.738382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.738363    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:16.739304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.739286    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:16:16.774947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.774914    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-8ed324"]
Oct 30 21:16:16.780378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.780349    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-8ed324"]
Oct 30 21:16:16.798372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.798262    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podStartSLOduration=1.798246114 podStartE2EDuration="1.798246114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:16.794267887 +0000 UTC m=+959.568667826" watchObservedRunningTime="2026-10-30 21:16:16.798246114 +0000 UTC m=+959.572646126"
Oct 30 21:16:16.951467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.951432    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-5dab06"]
Oct 30 21:16:16.952519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.952498    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2e95ec16-6cfe-4d74-a549-caed307d20c3" containerName="test-curl-7401-8ed324"
Oct 30 21:16:16.952584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.952524    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e95ec16-6cfe-4d74-a549-caed307d20c3" containerName="test-curl-7401-8ed324"
Oct 30 21:16:16.952733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.952719    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e95ec16-6cfe-4d74-a549-caed307d20c3" containerName="test-curl-7401-8ed324"
Oct 30 21:16:16.957136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.957115    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:16.969882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:16.966708    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:17.020334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.019519    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-5dab06"]
Oct 30 21:16:17.185549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.185517    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpdx4\" (UniqueName: \"kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4\") pod \"test-curl-7401-5dab06\" (UID: \"1601e60b-92a8-4352-894c-c63b660aa0f5\") " pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:17.215174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.215128    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:17.217264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.217222    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:17.217468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.217289    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:17.292983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.292943    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpdx4\" (UniqueName: \"kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4\") pod \"test-curl-7401-5dab06\" (UID: \"1601e60b-92a8-4352-894c-c63b660aa0f5\") " pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:17.318080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.318052    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpdx4\" (UniqueName: \"kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4\") pod \"test-curl-7401-5dab06\" (UID: \"1601e60b-92a8-4352-894c-c63b660aa0f5\") " pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:17.593856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.593092    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:17.746012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.745974    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7401-8ed324" podUID="2e95ec16-6cfe-4d74-a549-caed307d20c3" containerName="test-curl-7401-8ed324" containerID="cri-o://81f70e1ea18edddaa67a0b9403b5fb0cd78701771c41376dfa7e409d3a90d47e" gracePeriod=2
Oct 30 21:16:17.905808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.905781    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-5dab06"]
Oct 30 21:16:17.906367 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:17.906338    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1601e60b_92a8_4352_894c_c63b660aa0f5.slice/crio-beae17793255ea73c88b48a646369467db02e189b816a830b57dc2a077ea8f14 WatchSource:0}: Error finding container beae17793255ea73c88b48a646369467db02e189b816a830b57dc2a077ea8f14: Status 404 returned error can't find the container with id beae17793255ea73c88b48a646369467db02e189b816a830b57dc2a077ea8f14
Oct 30 21:16:17.996256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:17.996154    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:18.147529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.147496    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:18.212469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.212368    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:18.212469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.212429    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:18.305455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.305422    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:18.621503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.618070    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"]
Oct 30 21:16:18.718865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.689065    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:18.775127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.775097    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-92d5d744-dockercfg-rwjw5\""
Oct 30 21:16:18.780208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.780180    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-5dab06" event={"ID":"1601e60b-92a8-4352-894c-c63b660aa0f5","Type":"ContainerStarted","Data":"4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68"}
Oct 30 21:16:18.780405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.780389    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-5dab06" event={"ID":"1601e60b-92a8-4352-894c-c63b660aa0f5","Type":"ContainerStarted","Data":"beae17793255ea73c88b48a646369467db02e189b816a830b57dc2a077ea8f14"}
Oct 30 21:16:18.788917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.788872    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-92d5d744\""
Oct 30 21:16:18.842574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.842543    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"]
Oct 30 21:16:18.872549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.869376    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7401-5dab06" podStartSLOduration=2.86935694 podStartE2EDuration="2.86935694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:18.867624058 +0000 UTC m=+961.642023998" watchObservedRunningTime="2026-10-30 21:16:18.86935694 +0000 UTC m=+961.643756899"
Oct 30 21:16:18.944386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.944348    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume\") pod \"payload-processing-e2e-ait-92d5d744-779597976-7xjvh\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") " pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:18.944756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:18.944721    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmdrk\" (UniqueName: \"kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk\") pod \"payload-processing-e2e-ait-92d5d744-779597976-7xjvh\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") " pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:19.046406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.046363    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume\") pod \"payload-processing-e2e-ait-92d5d744-779597976-7xjvh\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") " pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:19.046586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.046547    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmdrk\" (UniqueName: \"kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk\") pod \"payload-processing-e2e-ait-92d5d744-779597976-7xjvh\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") " pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:19.047102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.047083    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume\") pod \"payload-processing-e2e-ait-92d5d744-779597976-7xjvh\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") " pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:19.066324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.066294    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmdrk\" (UniqueName: \"kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk\") pod \"payload-processing-e2e-ait-92d5d744-779597976-7xjvh\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") " pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:19.163797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.163708    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:19.318041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.308096    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:19.318041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.308157    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:19.410201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.400524    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:19.715155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.715125    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"]
Oct 30 21:16:19.717534 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:19.717507    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355aed7e_63ba_4a6e_b086_20e0b2aac940.slice/crio-ff271221d2972bea0c4d1d0f77821b258d52ff6c5e108c4ee2818ed6222ab187 WatchSource:0}: Error finding container ff271221d2972bea0c4d1d0f77821b258d52ff6c5e108c4ee2818ed6222ab187: Status 404 returned error can't find the container with id ff271221d2972bea0c4d1d0f77821b258d52ff6c5e108c4ee2818ed6222ab187
Oct 30 21:16:19.812117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.811237    4549 generic.go:356] "Generic (PLEG): container finished" podID="2e95ec16-6cfe-4d74-a549-caed307d20c3" containerID="81f70e1ea18edddaa67a0b9403b5fb0cd78701771c41376dfa7e409d3a90d47e" exitCode=137
Oct 30 21:16:19.814322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.814290    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh" event={"ID":"355aed7e-63ba-4a6e-b086-20e0b2aac940","Type":"ContainerStarted","Data":"ff271221d2972bea0c4d1d0f77821b258d52ff6c5e108c4ee2818ed6222ab187"}
Oct 30 21:16:19.978992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:19.968093    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:20.041537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.039957    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5\" (UniqueName: \"kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5\") pod \"2e95ec16-6cfe-4d74-a549-caed307d20c3\" (UID: \"2e95ec16-6cfe-4d74-a549-caed307d20c3\") "
Oct 30 21:16:20.041791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.041753    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-5dab06"]
Oct 30 21:16:20.043166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.043135    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:20.044675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.044652    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-5dab06"]
Oct 30 21:16:20.046862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.046816    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5" pod "2e95ec16-6cfe-4d74-a549-caed307d20c3" (UID: "2e95ec16-6cfe-4d74-a549-caed307d20c3"). InnerVolumeSpecName "kube-api-access-vw8j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:16:20.134240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.134207    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"]
Oct 30 21:16:20.135020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.135001    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" containerName="test-curl-7401-5dab06"
Oct 30 21:16:20.135094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.135022    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" containerName="test-curl-7401-5dab06"
Oct 30 21:16:20.135158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.135148    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" containerName="test-curl-7401-5dab06"
Oct 30 21:16:20.138788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.138764    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.142439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.142414    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw8j5\" (UniqueName: \"kubernetes.io/projected/2e95ec16-6cfe-4d74-a549-caed307d20c3-kube-api-access-vw8j5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:16:20.158884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.158853    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"]
Oct 30 21:16:20.161396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.159816    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4b62455c\""
Oct 30 21:16:20.204650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.201373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:20.212614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.212580    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:20.212775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.212639    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:20.244161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.244077    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-458gv\" (UniqueName: \"kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv\") pod \"payload-processing-e2e-ait-4b62455c-5945779696-2zmk7\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") " pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.244401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.244378    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume\") pod \"payload-processing-e2e-ait-4b62455c-5945779696-2zmk7\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") " pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.361854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.361307    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume\") pod \"payload-processing-e2e-ait-4b62455c-5945779696-2zmk7\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") " pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.361854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.361446    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-458gv\" (UniqueName: \"kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv\") pod \"payload-processing-e2e-ait-4b62455c-5945779696-2zmk7\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") " pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.396387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.387737    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume\") pod \"payload-processing-e2e-ait-4b62455c-5945779696-2zmk7\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") " pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.462860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.454667    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-458gv\" (UniqueName: \"kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv\") pod \"payload-processing-e2e-ait-4b62455c-5945779696-2zmk7\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") " pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.462860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.459458    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:20.839975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.838980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:20.850120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.850081    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh" event={"ID":"355aed7e-63ba-4a6e-b086-20e0b2aac940","Type":"ContainerStarted","Data":"f412683375898d09b6852fb4f74eaa2b18c012fcc489ab140df7a0b10242559a"}
Oct 30 21:16:20.850901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.850880    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:20.852695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.852669    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7401-5dab06" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" containerName="test-curl-7401-5dab06" containerID="cri-o://4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68" gracePeriod=2
Oct 30 21:16:20.853138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.853111    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:20.853501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:20.853460    4549 scope.go:122] "RemoveContainer" containerID="81f70e1ea18edddaa67a0b9403b5fb0cd78701771c41376dfa7e409d3a90d47e"
Oct 30 21:16:20.932018 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:20.931959    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-5dab06\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:20.946985 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:20.946923    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-8ed324\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="2e95ec16-6cfe-4d74-a549-caed307d20c3" pod="opendatahub/test-curl-7401-8ed324"
Oct 30 21:16:20.961112 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:20.961075    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-5dab06\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:21.070025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.069980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.166955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.166693    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh" podStartSLOduration=3.166665885 podStartE2EDuration="3.166665885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:20.918320166 +0000 UTC m=+963.692720134" watchObservedRunningTime="2026-10-30 21:16:21.166665885 +0000 UTC m=+963.941065837"
Oct 30 21:16:21.196384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.196254    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"]
Oct 30 21:16:21.212924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.212888    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:21.235474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.233125    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:21.285990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.277889    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.432955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.428292    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e95ec16-6cfe-4d74-a549-caed307d20c3" path="/var/lib/kubelet/pods/2e95ec16-6cfe-4d74-a549-caed307d20c3/volumes"
Oct 30 21:16:21.705806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.705724    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.860865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.860808    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" event={"ID":"7af8a814-79d1-419d-b8c8-3665e06ce58b","Type":"ContainerStarted","Data":"4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e"}
Oct 30 21:16:21.861339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.860897    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" event={"ID":"7af8a814-79d1-419d-b8c8-3665e06ce58b","Type":"ContainerStarted","Data":"b00ab62cd5256751613c327044faf7192c33d11528ea34bccbf811a5318a604d"}
Oct 30 21:16:21.861339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.860926    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:21.923824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:21.923773    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" podStartSLOduration=1.9237563149999999 podStartE2EDuration="1.923756315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:21.92088569 +0000 UTC m=+964.695285619" watchObservedRunningTime="2026-10-30 21:16:21.923756315 +0000 UTC m=+964.698156254"
Oct 30 21:16:22.239645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:22.239306    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:22.239645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:22.239365    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:22.905851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:22.905806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:23.213170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.213131    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 21:16:23.213365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.213198    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 21:16:23.261203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.261179    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:23.334735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.334694    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4\" (UniqueName: \"kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4\") pod \"1601e60b-92a8-4352-894c-c63b660aa0f5\" (UID: \"1601e60b-92a8-4352-894c-c63b660aa0f5\") "
Oct 30 21:16:23.337286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.337252    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4" pod "1601e60b-92a8-4352-894c-c63b660aa0f5" (UID: "1601e60b-92a8-4352-894c-c63b660aa0f5"). InnerVolumeSpecName "kube-api-access-bpdx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:16:23.348508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.348484    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1601e60b-92a8-4352-894c-c63b660aa0f5" path="/var/lib/kubelet/pods/1601e60b-92a8-4352-894c-c63b660aa0f5/volumes"
Oct 30 21:16:23.437653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.437558    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpdx4\" (UniqueName: \"kubernetes.io/projected/1601e60b-92a8-4352-894c-c63b660aa0f5-kube-api-access-bpdx4\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:16:23.876565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.876526    4549 generic.go:356] "Generic (PLEG): container finished" podID="1601e60b-92a8-4352-894c-c63b660aa0f5" containerID="4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68" exitCode=137
Oct 30 21:16:23.876743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.876568    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-5dab06"
Oct 30 21:16:23.876743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.876611    4549 scope.go:122] "RemoveContainer" containerID="4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68"
Oct 30 21:16:23.886309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.886287    4549 scope.go:122] "RemoveContainer" containerID="4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68"
Oct 30 21:16:23.886614 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:23.886596    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68\": container with ID starting with 4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68 not found: ID does not exist" containerID="4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68"
Oct 30 21:16:23.886667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:23.886624    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68"} err="failed to get container status \"4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68\": rpc error: code = NotFound desc = could not find container \"4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68\": container with ID starting with 4675e9f417ee9067d1d8dcfb314db8276e4aa469057eda9fafdd4b2caaaa4b68 not found: ID does not exist"
Oct 30 21:16:24.517380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:24.517342    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:24.720442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:24.720160    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:24.721442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:24.721265    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:24.726180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:24.726107    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:16:26.035524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:26.035489    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:26.574368 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:26.574336    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:16:26.740385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:26.740352    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:16:27.593720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:27.593676    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:27.737855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:27.737798    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:28.075217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:28.075186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:28.226511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:28.226477    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:29.347015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:29.346975    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:30.838951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.838908    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"]
Oct 30 21:16:30.851845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.850738    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.862802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.862770    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"]
Oct 30 21:16:30.868282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.868235    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.884444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.884414    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"]
Oct 30 21:16:30.900879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.900850    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"]
Oct 30 21:16:30.948633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.948570    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85kpj\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.948882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.948786    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.949080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.948920    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.948996    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949094    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949137    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949176    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949207    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949230    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.949362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949277    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcfsf\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949308    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.949362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949349    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:30.949500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949374    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.949500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949461    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.949605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949511    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.949997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.949973    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.950067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.950022    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:30.950067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:30.950058    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.036250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.036209    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"]
Oct 30 21:16:31.040639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.040617    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.051459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051433    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85kpj\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051511    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051555    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051589    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051614    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051630    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051651    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051669    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051685    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051716    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcfsf\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051733    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051758    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051775    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051875    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051908    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.051930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051936    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.052378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051959    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.052378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.051980    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.052378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.052339    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.053505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.053179    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.053815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.053782    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.054044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.054024    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.054429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.054405    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.054728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.054704    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.055290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.055261    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.055565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.055543    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.056248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.056225    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.056531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.056507    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.056610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.056512    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.057357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.057328    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.058843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.058806    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.059789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.059764    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.065443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.065420    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"]
Oct 30 21:16:31.080558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.080527    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.083098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.083070    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85kpj\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.083519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.083500    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.085052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.085031    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcfsf\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf\") pod \"e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") " pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.105549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.105475    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-92d5d744-serving-cert\""
Oct 30 21:16:31.153820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.153779    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cmld\" (UniqueName: \"kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.154038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.154007    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.154104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.154046    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.154249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.154235    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.167542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.167508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:31.197255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.197223    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:31.201366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.201335    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:31.256186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.255323    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cmld\" (UniqueName: \"kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.256186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.255528    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.256186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.255587    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.256186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.255780    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.256592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.256574    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.258690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.258670    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.259032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.259017    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.321066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.293549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:31.460194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.459683    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cmld\" (UniqueName: \"kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld\") pod \"maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.475106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.475052    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:31.736602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.736573    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"]
Oct 30 21:16:31.737231 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:31.737204    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6557f057_f9a4_4e7d_93a8_106f9d6248a3.slice/crio-6ef772087a3de8342adb842753182080fe53057545c68eada72ea11ac1e2b30f WatchSource:0}: Error finding container 6ef772087a3de8342adb842753182080fe53057545c68eada72ea11ac1e2b30f: Status 404 returned error can't find the container with id 6ef772087a3de8342adb842753182080fe53057545c68eada72ea11ac1e2b30f
Oct 30 21:16:31.740151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.740116    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.740250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.740186    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.740250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.740219    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.771588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.771562    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"]
Oct 30 21:16:31.773137 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:31.773115    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebd63a4_8699_4329_9480_3e022170dee4.slice/crio-e3a1e0d17733f3f8d26ee59c9187dba850878484fd8a301aabf6349011c847bc WatchSource:0}: Error finding container e3a1e0d17733f3f8d26ee59c9187dba850878484fd8a301aabf6349011c847bc: Status 404 returned error can't find the container with id e3a1e0d17733f3f8d26ee59c9187dba850878484fd8a301aabf6349011c847bc
Oct 30 21:16:31.776081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.776042    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.776157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.776128    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.776195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.776175    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.865474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.865442    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:16:31.878907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.878841    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"]
Oct 30 21:16:31.887649 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:31.887622    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5707ae75_58be_4938_adc4_2c0afe18422e.slice/crio-dd9945c449b06742ef49b7152b4af911e7e740138512a7bef05920f88aaa2c84 WatchSource:0}: Error finding container dd9945c449b06742ef49b7152b4af911e7e740138512a7bef05920f88aaa2c84: Status 404 returned error can't find the container with id dd9945c449b06742ef49b7152b4af911e7e740138512a7bef05920f88aaa2c84
Oct 30 21:16:31.952639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.951112    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm" event={"ID":"5707ae75-58be-4938-adc4-2c0afe18422e","Type":"ContainerStarted","Data":"dd9945c449b06742ef49b7152b4af911e7e740138512a7bef05920f88aaa2c84"}
Oct 30 21:16:31.955031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.954724    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" event={"ID":"7ebd63a4-8699-4329-9480-3e022170dee4","Type":"ContainerStarted","Data":"8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48"}
Oct 30 21:16:31.955031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.954782    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" event={"ID":"7ebd63a4-8699-4329-9480-3e022170dee4","Type":"ContainerStarted","Data":"e3a1e0d17733f3f8d26ee59c9187dba850878484fd8a301aabf6349011c847bc"}
Oct 30 21:16:31.959784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.958027    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" event={"ID":"6557f057-f9a4-4e7d-93a8-106f9d6248a3","Type":"ContainerStarted","Data":"b9326704c2e6660318ce9f1ef76978e94dcfe4d63ca5e6c1bb9c8a29f00d0018"}
Oct 30 21:16:31.959784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:31.958075    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" event={"ID":"6557f057-f9a4-4e7d-93a8-106f9d6248a3","Type":"ContainerStarted","Data":"6ef772087a3de8342adb842753182080fe53057545c68eada72ea11ac1e2b30f"}
Oct 30 21:16:32.000141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.000024    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podStartSLOduration=2.000005598 podStartE2EDuration="2.000005598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:31.996499186 +0000 UTC m=+974.770899136" watchObservedRunningTime="2026-10-30 21:16:32.000005598 +0000 UTC m=+974.774405563"
Oct 30 21:16:32.107267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.106917    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podStartSLOduration=2.106892866 podStartE2EDuration="2.106892866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:32.05801544 +0000 UTC m=+974.832415381" watchObservedRunningTime="2026-10-30 21:16:32.106892866 +0000 UTC m=+974.881292806"
Oct 30 21:16:32.199576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.197472    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:32.199989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.199960    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:32.200195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.200162    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:32.207032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.207005    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:32.207355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.207331    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:32.207505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.207482    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:32.910868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.909079    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:16:32.991897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.990055    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm" event={"ID":"5707ae75-58be-4938-adc4-2c0afe18422e","Type":"ContainerStarted","Data":"048dd2ce8e13c9f3eb2ef2978e3c64acf3400e7fb4d36d74d528cc68db39a72c"}
Oct 30 21:16:32.991897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:32.991293    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:33.043746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:33.043711    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:33.211970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:33.201650    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:33.212285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:33.212242    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:33.213193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:33.213166    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:33.213355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:33.213334    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:33.234881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:33.223396    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm" podStartSLOduration=1.625441753 podStartE2EDuration="2.223369232s" totalImagesPullingTime="597.927479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:33.15828076 +0000 UTC m=+975.932680702" watchObservedRunningTime="2026-10-30 21:16:33.223369232 +0000 UTC m=+975.997769170"
Oct 30 21:16:34.205858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:34.202138    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:34.205858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:34.202200    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:34.205858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:34.202616    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:34.205858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:34.202649    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:34.929863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:34.929677    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:35.089859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:35.088928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:35.205345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:35.205245    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:35.206590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:35.206554    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:35.207522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:35.205814    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:35.207684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:35.207662    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:35.703857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:35.697886    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:36.212859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:36.212492    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:36.212859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:36.212559    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:36.213527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:36.213248    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:36.213527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:36.213288    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:37.221859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:37.205308    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:37.221859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:37.205377    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:37.221859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:37.206776    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:37.221859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:37.206820    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:37.239955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:37.239925    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.842515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:37.842475    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:38.215958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:38.213943    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:38.215958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:38.214014    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:38.225286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:38.221248    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:38.225286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:38.221309    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:38.367513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:38.366208    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:38.968150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:38.965814    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:39.195252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:39.151182    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:39.223164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:39.209938    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:39.223430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:39.223401    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:39.224981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:39.224947    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:39.225248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:39.225220    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:39.714751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:39.714698    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:40.024715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.024620    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:40.120857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.117088    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:40.187856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.181939    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:40.187856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.183292    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:16:40.209643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.209603    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:40.211500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.211468    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:40.223962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.211407    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:40.224251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.224223    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:40.242031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.241984    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:40.581859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.570934    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:40.643570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.643526    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:16:40.643917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.643879    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:40.747720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.747683    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:40.757967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.757919    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:40.877547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.876981    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:40.877547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:40.877054    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:41.126133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.126079    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:41.127226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.127199    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:16:41.224586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.186481    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:41.224586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.187360    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:16:41.224586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.200928    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:41.224586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.200984    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:41.224586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.202014    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:41.224586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.202058    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:41.423424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.423373    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:16:41.673858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.666627    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:41.673858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.669625    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:41.721091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:41.721031    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:42.249890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.246867    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:42.265200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.265160    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:42.290110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.290073    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:42.290388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.290356    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:42.316860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.291048    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:42.317708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.317665    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:42.344300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.344249    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:16:42.344611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.344592    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:16:42.360605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.360571    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:16:42.399852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.386176    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:42.440981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.440939    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:42.671858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.667973    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:42.706960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.674938    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:16:42.780861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.773235    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:42.790401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.790358    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:16:42.814901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.810997    4549 patch_prober.go:29] interesting pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:42.831067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:42.818925    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:43.474496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:43.450099    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:43.474496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:43.450248    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:43.508294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:43.508250    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:43.609859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:43.593414    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:43.847615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:43.535303    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.226204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.226095    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:16:44.226463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.226429    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:16:44.256927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.237229    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:16:44.303939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.254894    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:44.306125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.254934    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:44.306570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.258469    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:44.306781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.306743    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:44.357895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.309390    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 21:16:44.570103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.570050    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 21:16:44.577898 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:44.476260    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.139s"
Oct 30 21:16:44.604339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.604296    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.638864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.615057    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.737259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:44.737213    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.188383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.187929    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.306999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.306949    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.307770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.307257    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.307962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.307289    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.699988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.564558    4549 patch_prober.go:29] interesting pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:45.699988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.564621    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.699988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.663312    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.699988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.663681    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.760859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:45.756925    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:16:46.199984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.199946    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:46.225716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.224421    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:46.258862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.257321    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:46.258862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.257402    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.258862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.258058    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:46.258862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.258114    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.311989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.311943    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.315856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.312631    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:46.316449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.316422    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.670003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.669946    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.670671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.670638    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:46.758868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:46.757456    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.317854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.308064    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:47.317854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.308146    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.317854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.313905    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.318968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.318912    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.319998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.319955    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.378049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.362946    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:47.378049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.363017    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.634990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.609398    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:47.756987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.710916    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:16:47.757572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.710959    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:47.780284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.758034    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.972456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:47.955933    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": context deadline exceeded"
Oct 30 21:16:48.252421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.250938    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.335497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.334147    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.335497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.334608    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:48.444859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.335738    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.444859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.358269    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:48.444859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.399906    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.444859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.384924    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:48.444859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.402312    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.446684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.446437    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.548929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.547859    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:48.732395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.731684    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:48.763931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.763659    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:16:48.811913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.783444    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.830878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:48.830799    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.341853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.338175    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.341853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.338616    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.424450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.424396    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.444693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.444658    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:49.445027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.444979    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.536007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.535960    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:49.536315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.536279    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.765067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.764961    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.893506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.893455    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.924355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:49.924292    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.379981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.379916    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.403748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.380260    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.441879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.436944    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.472788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.472744    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.525390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.525355    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:50.525665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.525639    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.553199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.552971    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:50.553199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.553037    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.823867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.819916    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.897985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.897935    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.918915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.916455    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:16:50.999607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:50.999580    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:16:51.015868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.015806    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:16:51.091278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.091194    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:16:51.106244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.104084    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:16:51.125806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.125774    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:16:51.151847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.151555    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.368529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.368439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.473195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.473160    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"]
Oct 30 21:16:51.484352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.484304    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:51.498922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.498891    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"]
Oct 30 21:16:51.499403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.499367    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1f954ca9\""
Oct 30 21:16:51.499624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.499609    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1f954ca9-dockercfg-bt7mp\""
Oct 30 21:16:51.502565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.502539    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.524285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.524253    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"]
Oct 30 21:16:51.532548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.532519    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:51.561946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.561908    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"]
Oct 30 21:16:51.562702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.562674    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume\") pod \"payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") " pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:51.562971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.562949    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4xp8\" (UniqueName: \"kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8\") pod \"payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") " pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:51.563293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.563262    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpt4r\" (UniqueName: \"kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r\") pod \"payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:51.563468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.563450    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:51.589415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.589365    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:51.589740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.589688    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:51.597271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.597230    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:51.597469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.597291    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:51.673906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.663728    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.715856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.706536    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:51.747864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.726099    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:16:51.798913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.749608    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume\") pod \"payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") " pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:51.819035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.818999    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4xp8\" (UniqueName: \"kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8\") pod \"payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") " pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:51.836877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.833072    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpt4r\" (UniqueName: \"kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r\") pod \"payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:51.881248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.803193    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume\") pod \"payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") " pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:51.998672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.987048    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:51.998672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:51.996569    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:52.123852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.108112    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4xp8\" (UniqueName: \"kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8\") pod \"payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") " pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:52.145276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.143821    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:52.227411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.227377    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpt4r\" (UniqueName: \"kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r\") pod \"payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:52.387719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.387685    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:52.394768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.394742    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:16:52.453043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.453012    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:52.499688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.498812    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"]
Oct 30 21:16:52.772073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:52.772044    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"]
Oct 30 21:16:52.772565 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:16:52.772542    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1476163b_023f_43b3_8947_f778ba1aaed0.slice/crio-6312a6147379a9a14072cd01990749118fe5c5c00327c06b74ef159c3fac7aa5 WatchSource:0}: Error finding container 6312a6147379a9a14072cd01990749118fe5c5c00327c06b74ef159c3fac7aa5: Status 404 returned error can't find the container with id 6312a6147379a9a14072cd01990749118fe5c5c00327c06b74ef159c3fac7aa5
Oct 30 21:16:53.129820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.129790    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:53.227209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.226643    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx" event={"ID":"1476163b-023f-43b3-8947-f778ba1aaed0","Type":"ContainerStarted","Data":"d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5"}
Oct 30 21:16:53.227209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.226712    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx" event={"ID":"1476163b-023f-43b3-8947-f778ba1aaed0","Type":"ContainerStarted","Data":"6312a6147379a9a14072cd01990749118fe5c5c00327c06b74ef159c3fac7aa5"}
Oct 30 21:16:53.228722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.228049    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:16:53.230788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.230219    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w" event={"ID":"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8","Type":"ContainerStarted","Data":"024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9"}
Oct 30 21:16:53.230788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.230281    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w" event={"ID":"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8","Type":"ContainerStarted","Data":"ab02cd0d09b54f6605a743e34d88a7e2b01faa4e6a2e2c3ac51d8da7a602cf87"}
Oct 30 21:16:53.281855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:53.281459    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx" podStartSLOduration=2.281437954 podStartE2EDuration="2.281437954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:53.278435786 +0000 UTC m=+996.052835727" watchObservedRunningTime="2026-10-30 21:16:53.281437954 +0000 UTC m=+996.055837894"
Oct 30 21:16:54.238318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:54.238287    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:16:54.730612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:54.730580    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:56.327025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:56.326993    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:56.576487 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:16:56.576443    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:16:57.735609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:57.735574    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:58.012910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:58.011072    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:58.164786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:58.164728    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:58.982698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:58.982656    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.136661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:59.136609    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.328191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:59.328161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.859854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:16:59.852557    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:01.510720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:01.510679    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:03.213442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:03.213402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:05.072780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:05.072745    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:05.259088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:05.259057    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:17:05.259650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:05.259623    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:17:05.301924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:05.301872    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w" podStartSLOduration=14.301819548 podStartE2EDuration="14.301819548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:53.32522452 +0000 UTC m=+996.099624458" watchObservedRunningTime="2026-10-30 21:17:05.301819548 +0000 UTC m=+1008.076219540"
Oct 30 21:17:05.682076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:05.682035    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:06.869038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:06.869003    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:08.596754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:08.596719    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:09.977404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:09.977366    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"]
Oct 30 21:17:09.982406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:09.982384    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.001624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.001594    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"]
Oct 30 21:17:10.002299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.002276    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.002419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.002398    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.002575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.002557    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.002685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.002667    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5zrj\" (UniqueName: \"kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.008621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.008590    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1f954ca9-serving-cert\""
Oct 30 21:17:10.085209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.085172    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:10.104224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.104194    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.104398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.104289    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.104453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.104411    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.104502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.104487    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5zrj\" (UniqueName: \"kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.104580 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:10.104559    4549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-serving-cert: secret "maas-api-e2e-mt-1f954ca9-serving-cert" not found
Oct 30 21:17:10.104648 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:10.104636    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls podName:2a74a760-d184-4219-a631-7a07cb625c19 nodeName:}" failed. No retries permitted until 2026-10-30 21:17:10.604614105 +0000 UTC m=+1013.379014037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls") pod "maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" (UID: "2a74a760-d184-4219-a631-7a07cb625c19") : secret "maas-api-e2e-mt-1f954ca9-serving-cert" not found
Oct 30 21:17:10.105055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.105036    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.107075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.107058    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.126552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.126524    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5zrj\" (UniqueName: \"kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.219863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.216916    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:10.633274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.633233    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.636130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.636111    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\") pod \"maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:10.682136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.682107    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:10.898610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:10.898518    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:11.463194 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:17:11.463149    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a74a760_d184_4219_a631_7a07cb625c19.slice/crio-256e501bd3d0c43061eaca4f7383cbb7c433c1a7e3174582b24be29258f7d0f0 WatchSource:0}: Error finding container 256e501bd3d0c43061eaca4f7383cbb7c433c1a7e3174582b24be29258f7d0f0: Status 404 returned error can't find the container with id 256e501bd3d0c43061eaca4f7383cbb7c433c1a7e3174582b24be29258f7d0f0
Oct 30 21:17:11.499244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:11.497614    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"]
Oct 30 21:17:11.508386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:11.508355    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:11.916341 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:11.916308    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:12.469641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:12.469595    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" event={"ID":"2a74a760-d184-4219-a631-7a07cb625c19","Type":"ContainerStarted","Data":"7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92"}
Oct 30 21:17:12.470107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:12.469664    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:12.470107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:12.469681    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" event={"ID":"2a74a760-d184-4219-a631-7a07cb625c19","Type":"ContainerStarted","Data":"256e501bd3d0c43061eaca4f7383cbb7c433c1a7e3174582b24be29258f7d0f0"}
Oct 30 21:17:12.510073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:12.510008    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" podStartSLOduration=3.120040904 podStartE2EDuration="3.509988671s" totalImagesPullingTime="389.947767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:12.509371642 +0000 UTC m=+1015.283771580" watchObservedRunningTime="2026-10-30 21:17:12.509988671 +0000 UTC m=+1015.284388618"
Oct 30 21:17:13.564161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:13.564127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:14.238733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:14.238698    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"]
Oct 30 21:17:14.364841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:14.364787    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:14.529117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:14.529031    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:15.571852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:15.565385    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:17.607699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:17.606998    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.695940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:17.693360    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:17:17.817868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:17.813005    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.907374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:17.907213    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:17.941819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:17.941763    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:18.239046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.238937    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:18.320812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.320762    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:17:18.321303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.320910    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:18.446952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.446798    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": context deadline exceeded"
Oct 30 21:17:18.566919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.563081    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:17:18.635858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.618324    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:17:18.635858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.619078    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:17:18.667026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.666980    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:17:18.697976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.697920    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:17:18.707861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.704476    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:17:18.814016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.813968    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:18.845555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:18.845458    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:17:19.243910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.240204    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9" podUID="befc60ca-1fd8-429d-bc60-bdda7c61bca1" containerName="payload-pre-processing" containerID="cri-o://7bf97fb65417454f09d1febdc16f91445032e86c9234f0b06b49bed89bbb330d" gracePeriod=25
Oct 30 21:17:19.475097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.470640    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:17:19.475097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.470927    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:17:19.709858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.701194    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:19.709858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.701366    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:17:19.710631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.710585    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:17:19.760859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:19.756921    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:17:20.143912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.106624    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:17:20.442643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.442525    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:20.508856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.474013    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:20.821719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.544474    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:20.831049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.544976    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:20.842871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.801901    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:17:20.843795 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:20.810117    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.409s"
Oct 30 21:17:20.873856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.844059    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:17:20.874190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.874164    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:17:20.874461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.874445    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:17:20.879258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:20.870071    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:21.147677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.143635    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:21.147677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.145096    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:17:21.791847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.791786    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:21.853864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.845996    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:17:21.953923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.859171    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:21.953923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.898953    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9" event={"ID":"befc60ca-1fd8-429d-bc60-bdda7c61bca1","Type":"ContainerDied","Data":"7bf97fb65417454f09d1febdc16f91445032e86c9234f0b06b49bed89bbb330d"}
Oct 30 21:17:21.953923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.874252    4549 generic.go:356] "Generic (PLEG): container finished" podID="befc60ca-1fd8-429d-bc60-bdda7c61bca1" containerID="7bf97fb65417454f09d1febdc16f91445032e86c9234f0b06b49bed89bbb330d" exitCode=0
Oct 30 21:17:21.953923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.899206    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:17:21.953923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.864058    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:17:21.958354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.958317    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"]
Oct 30 21:17:21.976037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:21.975646    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:22.153857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.149935    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:22.223547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.154989    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:22.348756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.348693    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": context deadline exceeded"
Oct 30 21:17:22.349147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.349108    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t" podUID="fd4d565e-8ace-4a8a-a8fb-114a90e3f64c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:22.846798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.846739    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:22.959448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.959405    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:17:22.960918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.960889    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:17:22.979874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:22.970981    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:23.293910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:23.038063    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:17:23.703045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:23.189185    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:23.703475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:23.265947    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:23.749352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:23.749296    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct" podUID="569e7b40-df79-438b-a350-7f4cf8202c9c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:24.287857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.276769    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:24.350934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.309417    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" podUID="7af8a814-79d1-419d-b8c8-3665e06ce58b" containerName="payload-processing" probeResult="failure" output="timeout: health rpc did not complete within 1s"
Oct 30 21:17:24.378993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.378958    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:17:24.379299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.379281    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"]
Oct 30 21:17:24.702563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.702507    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:24.742252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.742210    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:17:24.832098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.741572    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:24.876726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.876679    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn" podUID="0f292b66-399c-4566-8921-3f04abdaed5a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:24.942866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.893051    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:17:24.971920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:24.938413    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:25.296012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.291490    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:17:25.329099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.316771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:25.433052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.432987    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": context deadline exceeded"
Oct 30 21:17:25.528450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.528415    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:25.582860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.574871    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:25.649888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.648260    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:17:25.701078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.700627    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"]
Oct 30 21:17:25.702236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.702211    4549 cpu_manager.go:414] "containerMap: removing container" podUID="befc60ca-1fd8-429d-bc60-bdda7c61bca1" containerName="payload-pre-processing"
Oct 30 21:17:25.703061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.702391    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="befc60ca-1fd8-429d-bc60-bdda7c61bca1" containerName="payload-pre-processing"
Oct 30 21:17:25.703061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.702615    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="befc60ca-1fd8-429d-bc60-bdda7c61bca1" containerName="payload-pre-processing"
Oct 30 21:17:25.707355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.707337    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:17:25.707531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.707516    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"]
Oct 30 21:17:25.707676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.707656    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9" event={"ID":"befc60ca-1fd8-429d-bc60-bdda7c61bca1","Type":"ContainerDied","Data":"c71032ade55136e77218a52c653c2a66863f77314954e4031ec267f8308709f3"}
Oct 30 21:17:25.707913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.707896    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:17:25.708048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.708036    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:17:25.709232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.709213    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.713840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.712470    4549 scope.go:122] "RemoveContainer" containerID="7bf97fb65417454f09d1febdc16f91445032e86c9234f0b06b49bed89bbb330d"
Oct 30 21:17:25.713840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.712959    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:25.713840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.713196    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:17:25.762770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.762740    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-399049f3-openshift-default-dockercfg-ztswq\""
Oct 30 21:17:25.766494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.766473    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:17:25.802863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.787463    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn\" (UniqueName: \"kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn\") pod \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") "
Oct 30 21:17:25.803527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.803494    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume\") pod \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\" (UID: \"befc60ca-1fd8-429d-bc60-bdda7c61bca1\") "
Oct 30 21:17:25.826415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.826374    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.826693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.826666    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p48b5\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.827160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.827135    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.827432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.827411    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.827653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.827634    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.827819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.827802    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.828023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.827999    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.833123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.833055    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.833294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.833275    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.835853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.830101    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume" pod "befc60ca-1fd8-429d-bc60-bdda7c61bca1" (UID: "befc60ca-1fd8-429d-bc60-bdda7c61bca1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:25.840016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.839989    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn" pod "befc60ca-1fd8-429d-bc60-bdda7c61bca1" (UID: "befc60ca-1fd8-429d-bc60-bdda7c61bca1"). InnerVolumeSpecName "kube-api-access-724vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:25.935482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935434    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.935685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935516    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.935685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935590    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.935685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935638    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.935909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935696    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.935909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935813    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.935909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935864    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.936070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.935993    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.936070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.936022    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p48b5\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.936223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.936168    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-724vn\" (UniqueName: \"kubernetes.io/projected/befc60ca-1fd8-429d-bc60-bdda7c61bca1-kube-api-access-724vn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:25.936223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.936190    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/befc60ca-1fd8-429d-bc60-bdda7c61bca1-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:25.937005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.936974    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.937436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.937410    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.937528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.937494    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.937654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.937629    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.938006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.937977    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.939305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.939281    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.939793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.939769    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.956366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.956334    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p48b5\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:25.956633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:25.956615    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token\") pod \"e2e-ait-399049f3-openshift-default-778658c8c-ltmlv\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") " pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:26.137691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.137597    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:26.165870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.165819    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"]
Oct 30 21:17:26.204455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.166589    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm" podUID="5707ae75-58be-4938-adc4-2c0afe18422e" containerName="maas-api" containerID="cri-o://048dd2ce8e13c9f3eb2ef2978e3c64acf3400e7fb4d36d74d528cc68db39a72c" gracePeriod=30
Oct 30 21:17:26.253200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.252524    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:26.766158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.766064    4549 generic.go:356] "Generic (PLEG): container finished" podID="5707ae75-58be-4938-adc4-2c0afe18422e" containerID="048dd2ce8e13c9f3eb2ef2978e3c64acf3400e7fb4d36d74d528cc68db39a72c" exitCode=0
Oct 30 21:17:26.768106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.766949    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm" event={"ID":"5707ae75-58be-4938-adc4-2c0afe18422e","Type":"ContainerDied","Data":"048dd2ce8e13c9f3eb2ef2978e3c64acf3400e7fb4d36d74d528cc68db39a72c"}
Oct 30 21:17:26.782659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.782626    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"
Oct 30 21:17:26.965862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:26.960898    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh" podUID="355aed7e-63ba-4a6e-b086-20e0b2aac940" containerName="payload-processing" containerID="cri-o://f412683375898d09b6852fb4f74eaa2b18c012fcc489ab140df7a0b10242559a" gracePeriod=25
Oct 30 21:17:27.291909 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:27.284279    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:17:27.413294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.392718    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.638999 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:17:27.631735    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a23f05c_f215_4618_9e40_1bab6a597276.slice/crio-250648f05824ded3a7a156ceba520a398594bcfb827cf52f9fc65746f6b74d5b WatchSource:0}: Error finding container 250648f05824ded3a7a156ceba520a398594bcfb827cf52f9fc65746f6b74d5b: Status 404 returned error can't find the container with id 250648f05824ded3a7a156ceba520a398594bcfb827cf52f9fc65746f6b74d5b
Oct 30 21:17:27.670998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.657482    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"]
Oct 30 21:17:27.670998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.658054    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9"]
Oct 30 21:17:27.670998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.658190    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.705494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.687096    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:27.705494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.687208    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:27.705494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.687276    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:27.794876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.792265    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:27.854703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.854576    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"]
Oct 30 21:17:27.865261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.865224    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:27.870734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.870657    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" event={"ID":"5a23f05c-f215-4618-9e40-1bab6a597276","Type":"ContainerStarted","Data":"250648f05824ded3a7a156ceba520a398594bcfb827cf52f9fc65746f6b74d5b"}
Oct 30 21:17:27.873028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.872801    4549 generic.go:356] "Generic (PLEG): container finished" podID="355aed7e-63ba-4a6e-b086-20e0b2aac940" containerID="f412683375898d09b6852fb4f74eaa2b18c012fcc489ab140df7a0b10242559a" exitCode=0
Oct 30 21:17:27.873028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:27.872970    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh" event={"ID":"355aed7e-63ba-4a6e-b086-20e0b2aac940","Type":"ContainerDied","Data":"f412683375898d09b6852fb4f74eaa2b18c012fcc489ab140df7a0b10242559a"}
Oct 30 21:17:28.037481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.037421    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:28.044405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.044381    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:17:28.132400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.118223    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld\" (UniqueName: \"kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld\") pod \"5707ae75-58be-4938-adc4-2c0afe18422e\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") "
Oct 30 21:17:28.132400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.118345    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca\") pod \"5707ae75-58be-4938-adc4-2c0afe18422e\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") "
Oct 30 21:17:28.132400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.118602    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls\") pod \"5707ae75-58be-4938-adc4-2c0afe18422e\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") "
Oct 30 21:17:28.132400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.118696    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs\") pod \"5707ae75-58be-4938-adc4-2c0afe18422e\" (UID: \"5707ae75-58be-4938-adc4-2c0afe18422e\") "
Oct 30 21:17:28.173072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.173033    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca" pod "5707ae75-58be-4938-adc4-2c0afe18422e" (UID: "5707ae75-58be-4938-adc4-2c0afe18422e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:28.195229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.195154    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs" pod "5707ae75-58be-4938-adc4-2c0afe18422e" (UID: "5707ae75-58be-4938-adc4-2c0afe18422e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:17:28.230900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.230817    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls" pod "5707ae75-58be-4938-adc4-2c0afe18422e" (UID: "5707ae75-58be-4938-adc4-2c0afe18422e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:17:28.240329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.235945    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld" pod "5707ae75-58be-4938-adc4-2c0afe18422e" (UID: "5707ae75-58be-4938-adc4-2c0afe18422e"). InnerVolumeSpecName "kube-api-access-8cmld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:28.326864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.304313    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cmld\" (UniqueName: \"kubernetes.io/projected/5707ae75-58be-4938-adc4-2c0afe18422e-kube-api-access-8cmld\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:28.326864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.304350    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5707ae75-58be-4938-adc4-2c0afe18422e-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:28.326864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.304366    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:28.326864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.304381    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5707ae75-58be-4938-adc4-2c0afe18422e-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:28.360661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.359980    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:17:28.410036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.389198    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"]
Oct 30 21:17:28.410036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.389479    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" podUID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" containerName="istio-proxy" containerID="cri-o://77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f" gracePeriod=30
Oct 30 21:17:28.433272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.433228    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk\" (UniqueName: \"kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk\") pod \"355aed7e-63ba-4a6e-b086-20e0b2aac940\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") "
Oct 30 21:17:28.433629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.433607    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume\") pod \"355aed7e-63ba-4a6e-b086-20e0b2aac940\" (UID: \"355aed7e-63ba-4a6e-b086-20e0b2aac940\") "
Oct 30 21:17:28.435780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.435468    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume" pod "355aed7e-63ba-4a6e-b086-20e0b2aac940" (UID: "355aed7e-63ba-4a6e-b086-20e0b2aac940"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:28.460256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.459632    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk" pod "355aed7e-63ba-4a6e-b086-20e0b2aac940" (UID: "355aed7e-63ba-4a6e-b086-20e0b2aac940"). InnerVolumeSpecName "kube-api-access-bmdrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:28.468222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.468058    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:28.538816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.538782    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmdrk\" (UniqueName: \"kubernetes.io/projected/355aed7e-63ba-4a6e-b086-20e0b2aac940-kube-api-access-bmdrk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:28.539064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.539049    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355aed7e-63ba-4a6e-b086-20e0b2aac940-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:28.633411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.633347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:28.904856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.899452    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" event={"ID":"5a23f05c-f215-4618-9e40-1bab6a597276","Type":"ContainerStarted","Data":"ebf2bb1770ad8aebd88708375190e1c6eba073b04b4011b7bae90abeea4c1a4f"}
Oct 30 21:17:28.904856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.902440    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh" event={"ID":"355aed7e-63ba-4a6e-b086-20e0b2aac940","Type":"ContainerDied","Data":"ff271221d2972bea0c4d1d0f77821b258d52ff6c5e108c4ee2818ed6222ab187"}
Oct 30 21:17:28.904856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.902498    4549 scope.go:122] "RemoveContainer" containerID="f412683375898d09b6852fb4f74eaa2b18c012fcc489ab140df7a0b10242559a"
Oct 30 21:17:28.904856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.902649    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"
Oct 30 21:17:28.938856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.930591    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm" event={"ID":"5707ae75-58be-4938-adc4-2c0afe18422e","Type":"ContainerDied","Data":"dd9945c449b06742ef49b7152b4af911e7e740138512a7bef05920f88aaa2c84"}
Oct 30 21:17:28.938856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:28.930741    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"
Oct 30 21:17:29.052137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.052015    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" podStartSLOduration=5.051994737 podStartE2EDuration="5.051994737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:28.9784842 +0000 UTC m=+1031.752884141" watchObservedRunningTime="2026-10-30 21:17:29.051994737 +0000 UTC m=+1031.826394677"
Oct 30 21:17:29.064920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.062924    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"]
Oct 30 21:17:29.083902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.068119    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh"]
Oct 30 21:17:29.151037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.136326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:29.151037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.139362    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:29.151037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.140679    4549 patch_prober.go:29] interesting pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 21:17:29.151037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.140732    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 21:17:29.176780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.162044    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"]
Oct 30 21:17:29.176780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.171617    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92d5d744-6894dfcbfb-cxkvm"]
Oct 30 21:17:29.219895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.219536    4549 scope.go:122] "RemoveContainer" containerID="048dd2ce8e13c9f3eb2ef2978e3c64acf3400e7fb4d36d74d528cc68db39a72c"
Oct 30 21:17:29.351791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.351749    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="355aed7e-63ba-4a6e-b086-20e0b2aac940" path="/var/lib/kubelet/pods/355aed7e-63ba-4a6e-b086-20e0b2aac940/volumes"
Oct 30 21:17:29.352517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.352492    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5707ae75-58be-4938-adc4-2c0afe18422e" path="/var/lib/kubelet/pods/5707ae75-58be-4938-adc4-2c0afe18422e/volumes"
Oct 30 21:17:29.353130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.353111    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="befc60ca-1fd8-429d-bc60-bdda7c61bca1" path="/var/lib/kubelet/pods/befc60ca-1fd8-429d-bc60-bdda7c61bca1/volumes"
Oct 30 21:17:29.502642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.502542    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"]
Oct 30 21:17:29.503620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.503599    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5707ae75-58be-4938-adc4-2c0afe18422e" containerName="maas-api"
Oct 30 21:17:29.503620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.503620    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5707ae75-58be-4938-adc4-2c0afe18422e" containerName="maas-api"
Oct 30 21:17:29.503796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.503641    4549 cpu_manager.go:414] "containerMap: removing container" podUID="355aed7e-63ba-4a6e-b086-20e0b2aac940" containerName="payload-processing"
Oct 30 21:17:29.503796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.503650    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="355aed7e-63ba-4a6e-b086-20e0b2aac940" containerName="payload-processing"
Oct 30 21:17:29.503975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.503863    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5707ae75-58be-4938-adc4-2c0afe18422e" containerName="maas-api"
Oct 30 21:17:29.503975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.503877    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="355aed7e-63ba-4a6e-b086-20e0b2aac940" containerName="payload-processing"
Oct 30 21:17:29.510285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.510243    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.523263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.523231    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"]
Oct 30 21:17:29.532726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.532700    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c906b806-openshift-default-dockercfg-jccbc\""
Oct 30 21:17:29.572666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.572609    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.572984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.572884    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.572984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.572946    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.573145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.573001    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.573145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.573071    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.573145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.573113    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.573314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.573190    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t7gt\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.573314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.573247    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.573314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.573279    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.598561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.598501    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:29.686113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.681033    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t7gt\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.688339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.688306    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.689063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.689029    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.690133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.690112    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.690605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.690588    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.691178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.691151    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.691658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.691640    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.692181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.692160    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.692420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.688964    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.692420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.690482    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.692420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.689793    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.692630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.692442    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.692734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.692715    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.693088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.693059    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.751383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.739240    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.759907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.756544    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.759907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.757167    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:29.813821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.813780    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" podUID="7af8a814-79d1-419d-b8c8-3665e06ce58b" containerName="payload-processing" containerID="cri-o://4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e" gracePeriod=25
Oct 30 21:17:29.819037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.815246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:29.988927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:29.985023    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t7gt\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt\") pod \"e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") " pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:30.151625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.151381    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:30.158537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.156666    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:30.373367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.373340    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:17:30.395501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.395443    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:30.454862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.454484    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume\") pod \"7af8a814-79d1-419d-b8c8-3665e06ce58b\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") "
Oct 30 21:17:30.455781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.455244    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume" pod "7af8a814-79d1-419d-b8c8-3665e06ce58b" (UID: "7af8a814-79d1-419d-b8c8-3665e06ce58b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:30.455781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.455612    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv\" (UniqueName: \"kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv\") pod \"7af8a814-79d1-419d-b8c8-3665e06ce58b\" (UID: \"7af8a814-79d1-419d-b8c8-3665e06ce58b\") "
Oct 30 21:17:30.457152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.457130    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af8a814-79d1-419d-b8c8-3665e06ce58b-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:30.458355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.458317    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv" pod "7af8a814-79d1-419d-b8c8-3665e06ce58b" (UID: "7af8a814-79d1-419d-b8c8-3665e06ce58b"). InnerVolumeSpecName "kube-api-access-458gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:30.589955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.589901    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-458gv\" (UniqueName: \"kubernetes.io/projected/7af8a814-79d1-419d-b8c8-3665e06ce58b-kube-api-access-458gv\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:30.647358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.647215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:30.763391 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:17:30.763345    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46557a81_9192_41be_98e3_7f5043cfe6fe.slice/crio-eee8e0b54d6c7f791556e2472ec378c2197f5c56c88eefc63d5d3ee43d8f4452 WatchSource:0}: Error finding container eee8e0b54d6c7f791556e2472ec378c2197f5c56c88eefc63d5d3ee43d8f4452: Status 404 returned error can't find the container with id eee8e0b54d6c7f791556e2472ec378c2197f5c56c88eefc63d5d3ee43d8f4452
Oct 30 21:17:30.789324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.788952    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:30.789324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.789067    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:30.800960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.800351    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:30.826125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.823990    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"]
Oct 30 21:17:30.894792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:30.836542    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.402289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.402251    4549 generic.go:356] "Generic (PLEG): container finished" podID="7af8a814-79d1-419d-b8c8-3665e06ce58b" containerID="4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e" exitCode=0
Oct 30 21:17:31.402952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.402422    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"
Oct 30 21:17:31.409439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.409401    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" event={"ID":"46557a81-9192-41be-98e3-7f5043cfe6fe","Type":"ContainerStarted","Data":"eee8e0b54d6c7f791556e2472ec378c2197f5c56c88eefc63d5d3ee43d8f4452"}
Oct 30 21:17:31.409611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.409466    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" event={"ID":"7af8a814-79d1-419d-b8c8-3665e06ce58b","Type":"ContainerDied","Data":"4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e"}
Oct 30 21:17:31.409611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.409505    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:31.409611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.409572    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:17:31.409611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.409595    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7" event={"ID":"7af8a814-79d1-419d-b8c8-3665e06ce58b","Type":"ContainerDied","Data":"b00ab62cd5256751613c327044faf7192c33d11528ea34bccbf811a5318a604d"}
Oct 30 21:17:31.409882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.409624    4549 scope.go:122] "RemoveContainer" containerID="4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e"
Oct 30 21:17:31.428995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.428965    4549 scope.go:122] "RemoveContainer" containerID="4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e"
Oct 30 21:17:31.429331 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:31.429310    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e\": container with ID starting with 4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e not found: ID does not exist" containerID="4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e"
Oct 30 21:17:31.429402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.429342    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e"} err="failed to get container status \"4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e\": rpc error: code = NotFound desc = could not find container \"4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e\": container with ID starting with 4ccf04cc03772583cc1d9adfd0b93eea2e5573280b10b0e44492add9bec5089e not found: ID does not exist"
Oct 30 21:17:31.511109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.511045    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.537036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.536953    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"]
Oct 30 21:17:31.543871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.543776    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7"]
Oct 30 21:17:31.696233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.696146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.812858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.804613    4549 patch_prober.go:29] interesting pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:17:31.812858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.804679    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" podUID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:31.878451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.878323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.883206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.883166    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"]
Oct 30 21:17:31.884142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:31.884100    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" podUID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" containerName="istio-proxy" containerID="cri-o://f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206" gracePeriod=30
Oct 30 21:17:32.021912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.021755    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.438667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.438505    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" event={"ID":"46557a81-9192-41be-98e3-7f5043cfe6fe","Type":"ContainerStarted","Data":"1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9"}
Oct 30 21:17:32.492375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.485952    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" podStartSLOduration=3.485930834 podStartE2EDuration="3.485930834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:32.484471215 +0000 UTC m=+1035.258871155" watchObservedRunningTime="2026-10-30 21:17:32.485930834 +0000 UTC m=+1035.260330776"
Oct 30 21:17:32.572329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.572275    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.667710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.667663    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"]
Oct 30 21:17:32.668938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.668905    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7af8a814-79d1-419d-b8c8-3665e06ce58b" containerName="payload-processing"
Oct 30 21:17:32.668938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.668935    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7af8a814-79d1-419d-b8c8-3665e06ce58b" containerName="payload-processing"
Oct 30 21:17:32.669134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.669112    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7af8a814-79d1-419d-b8c8-3665e06ce58b" containerName="payload-processing"
Oct 30 21:17:32.674242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.674204    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.699320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.696789    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-399049f3\""
Oct 30 21:17:32.699320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.698159    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"]
Oct 30 21:17:32.711980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.711941    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.764326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.764230    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jrnf\" (UniqueName: \"kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf\") pod \"payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") " pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.765114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.765079    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume\") pod \"payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") " pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.783477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.783433    4549 patch_prober.go:29] interesting pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:17:32.783753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.783508    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" podUID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:32.858296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.858251    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.866781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.866733    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume\") pod \"payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") " pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.866997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.866918    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jrnf\" (UniqueName: \"kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf\") pod \"payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") " pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.867680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.867660    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume\") pod \"payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") " pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.901595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.901523    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jrnf\" (UniqueName: \"kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf\") pod \"payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") " pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:32.989946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:32.989791    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:33.016871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.016321    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:33.180163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.171063    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:33.214564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.214215    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:33.227088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.226985    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:33.443619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.443573    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7af8a814-79d1-419d-b8c8-3665e06ce58b" path="/var/lib/kubelet/pods/7af8a814-79d1-419d-b8c8-3665e06ce58b/volumes"
Oct 30 21:17:33.456497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.456283    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:33.458946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.458456    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:17:33.751158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:33.751117    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"]
Oct 30 21:17:33.753176 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:17:33.753133    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3373a9a_c86d_43b4_9fb7_18c075210666.slice/crio-06146a123a0a1730e515d8fca49ab32f2768b10b48dec09e6f6b2a6dee2b3422 WatchSource:0}: Error finding container 06146a123a0a1730e515d8fca49ab32f2768b10b48dec09e6f6b2a6dee2b3422: Status 404 returned error can't find the container with id 06146a123a0a1730e515d8fca49ab32f2768b10b48dec09e6f6b2a6dee2b3422
Oct 30 21:17:34.333308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.333272    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.543754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.543127    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h" event={"ID":"e3373a9a-c86d-43b4-9fb7-18c075210666","Type":"ContainerStarted","Data":"0b66ca25b80fb09dc76752fe9b7b205449eea35f46ff7f9e6a3a04dacdea5c37"}
Oct 30 21:17:34.543754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.543187    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h" event={"ID":"e3373a9a-c86d-43b4-9fb7-18c075210666","Type":"ContainerStarted","Data":"06146a123a0a1730e515d8fca49ab32f2768b10b48dec09e6f6b2a6dee2b3422"}
Oct 30 21:17:34.543754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.543241    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:34.590766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.590662    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h" podStartSLOduration=2.590647345 podStartE2EDuration="2.590647345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:34.585213014 +0000 UTC m=+1037.359612955" watchObservedRunningTime="2026-10-30 21:17:34.590647345 +0000 UTC m=+1037.365047283"
Oct 30 21:17:34.725430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.725370    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.890868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:34.890697    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:35.051700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:35.051599    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:36.352939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:36.352206    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:36.499375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:36.499320    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:37.419957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.419920    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:17:37.448638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.448598    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.448808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.448646    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.448808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.448757    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.448818    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.448931    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449078    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.449166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449105    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449159    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449185    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:37.449333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449231    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449290    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo\") pod \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\" (UID: \"439dd61b-80cd-4a64-b31c-4fd3c2023c6b\") "
Oct 30 21:17:37.449436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449336    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.449495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449460    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.449615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.449593    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.450788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.450669    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.450788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.450695    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.450788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.450711    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.450788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.450725    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.450788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.450738    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.451381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.451355    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.451819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.451798    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:17:37.452152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.452132    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "kube-api-access-rpgqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:37.452201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.452164    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token" pod "439dd61b-80cd-4a64-b31c-4fd3c2023c6b" (UID: "439dd61b-80cd-4a64-b31c-4fd3c2023c6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:37.477490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.477160    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:17:37.562470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562376    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562433    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562520    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562581    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562626    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562698    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562724    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562755    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.562856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.562812    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs\") pod \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\" (UID: \"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17\") "
Oct 30 21:17:37.563611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.563581    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpgqd\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-kube-api-access-rpgqd\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.563611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.563591    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.563611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.563611    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.563935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.563626    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.563935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.563638    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/439dd61b-80cd-4a64-b31c-4fd3c2023c6b-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.564078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.564047    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.564358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.564335    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.564446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.564402    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.564446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.564410    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:37.565912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.565794    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:37.567101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.567071    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:37.567200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.567131    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "kube-api-access-qdcxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:37.567246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.567194    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo" pod "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" (UID: "5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:17:37.665361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665327    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665358    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665368    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdcxg\" (UniqueName: \"kubernetes.io/projected/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-kube-api-access-qdcxg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665377    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665388    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665398    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665408    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665417    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.665600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.665425    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:17:37.697349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.697312    4549 generic.go:356] "Generic (PLEG): container finished" podID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" containerID="77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f" exitCode=0
Oct 30 21:17:37.697503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.697360    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"
Oct 30 21:17:37.697503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.697402    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" event={"ID":"439dd61b-80cd-4a64-b31c-4fd3c2023c6b","Type":"ContainerDied","Data":"77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f"}
Oct 30 21:17:37.697503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.697462    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh" event={"ID":"439dd61b-80cd-4a64-b31c-4fd3c2023c6b","Type":"ContainerDied","Data":"dc1d669a07337ebdcecba10a2c180620da49ff505c2fb3235ad17d26bf56b596"}
Oct 30 21:17:37.697503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.697486    4549 scope.go:122] "RemoveContainer" containerID="77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f"
Oct 30 21:17:37.699219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.699196    4549 generic.go:356] "Generic (PLEG): container finished" podID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" containerID="f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206" exitCode=0
Oct 30 21:17:37.699348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.699249    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"
Oct 30 21:17:37.699348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.699297    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" event={"ID":"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17","Type":"ContainerDied","Data":"f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206"}
Oct 30 21:17:37.699469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.699353    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz" event={"ID":"5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17","Type":"ContainerDied","Data":"1200b4b447a60de940ff501e4589cd013a9ea91f43c9b1faa24a4ea0bfbff6ba"}
Oct 30 21:17:37.709381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.709314    4549 scope.go:122] "RemoveContainer" containerID="77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f"
Oct 30 21:17:37.709689 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:37.709668    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f\": container with ID starting with 77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f not found: ID does not exist" containerID="77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f"
Oct 30 21:17:37.709758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.709699    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f"} err="failed to get container status \"77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f\": rpc error: code = NotFound desc = could not find container \"77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f\": container with ID starting with 77d36d96700f8c61f484cc9a69612549e3b570faf7843293cbe77a7d977c3d8f not found: ID does not exist"
Oct 30 21:17:37.709758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.709716    4549 scope.go:122] "RemoveContainer" containerID="f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206"
Oct 30 21:17:37.719910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.719887    4549 scope.go:122] "RemoveContainer" containerID="f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206"
Oct 30 21:17:37.720269 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:37.720237    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206\": container with ID starting with f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206 not found: ID does not exist" containerID="f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206"
Oct 30 21:17:37.720346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.720283    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206"} err="failed to get container status \"f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206\": rpc error: code = NotFound desc = could not find container \"f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206\": container with ID starting with f0924a843e05092588229ce13d784aaba5674b220cc59af2655a2367b4543206 not found: ID does not exist"
Oct 30 21:17:37.750305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.750266    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"]
Oct 30 21:17:37.754132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.754100    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh"]
Oct 30 21:17:37.786241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.786201    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"]
Oct 30 21:17:37.790387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:37.790356    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz"]
Oct 30 21:17:38.168875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:38.168161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:39.388284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:39.388241    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" path="/var/lib/kubelet/pods/439dd61b-80cd-4a64-b31c-4fd3c2023c6b/volumes"
Oct 30 21:17:39.389219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:39.389191    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" path="/var/lib/kubelet/pods/5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17/volumes"
Oct 30 21:17:40.271853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:40.265874    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:41.973859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:41.968709    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:42.659726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:42.649381    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:42.927923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:42.927795    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:43.090865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:43.085607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:43.463142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:43.463102    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:44.628077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:44.628042    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.107468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:46.107435    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.699778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:46.699748    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:17:47.719519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:47.719483    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.088964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.088927    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"]
Oct 30 21:17:49.089660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.089642    4549 cpu_manager.go:414] "containerMap: removing container" podUID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" containerName="istio-proxy"
Oct 30 21:17:49.089704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.089662    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" containerName="istio-proxy"
Oct 30 21:17:49.089704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.089693    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" containerName="istio-proxy"
Oct 30 21:17:49.089704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.089698    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" containerName="istio-proxy"
Oct 30 21:17:49.089807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.089797    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="439dd61b-80cd-4a64-b31c-4fd3c2023c6b" containerName="istio-proxy"
Oct 30 21:17:49.089856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.089811    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c10a9b9-7c4e-46b9-a4b2-6b7e3a6fcf17" containerName="istio-proxy"
Oct 30 21:17:49.093136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.093118    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.120884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.116128    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-399049f3-serving-cert\""
Oct 30 21:17:49.120884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.119234    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"]
Oct 30 21:17:49.141815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.141780    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.142135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.142090    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.142225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.142205    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.142481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.142462    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glnq\" (UniqueName: \"kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.183043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.183001    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.243766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.243733    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glnq\" (UniqueName: \"kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.243951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.243903    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.244080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.244062    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.244144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.244131    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.244711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.244681    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.246551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.246530    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.246658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.246598    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.264512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.264486    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glnq\" (UniqueName: \"kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq\") pod \"maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.347448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.347357    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.415105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.414726    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:49.498451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.498343    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.832436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.832403    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"]
Oct 30 21:17:49.833209 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:17:49.833184    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17614e48_9d90_452d_b5eb_2dc001b7830c.slice/crio-2ac09088e1cf9c299fd811077e143cbc2834c5d146956d1603b3605cc87092be WatchSource:0}: Error finding container 2ac09088e1cf9c299fd811077e143cbc2834c5d146956d1603b3605cc87092be: Status 404 returned error can't find the container with id 2ac09088e1cf9c299fd811077e143cbc2834c5d146956d1603b3605cc87092be
Oct 30 21:17:49.937608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:49.937573    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58" event={"ID":"17614e48-9d90-452d-b5eb-2dc001b7830c","Type":"ContainerStarted","Data":"2ac09088e1cf9c299fd811077e143cbc2834c5d146956d1603b3605cc87092be"}
Oct 30 21:17:50.897822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:50.897784    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:50.968643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:50.968603    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58" event={"ID":"17614e48-9d90-452d-b5eb-2dc001b7830c","Type":"ContainerStarted","Data":"855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936"}
Oct 30 21:17:50.970097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:50.970071    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:51.044250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:51.044167    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58" podStartSLOduration=1.717999745 podStartE2EDuration="2.044143543s" totalImagesPullingTime="326.143798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:51.032323735 +0000 UTC m=+1053.806723675" watchObservedRunningTime="2026-10-30 21:17:51.044143543 +0000 UTC m=+1053.818543483"
Oct 30 21:17:52.354863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.354819    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"]
Oct 30 21:17:52.358107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.358085    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.377520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.377497    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"]
Oct 30 21:17:52.384108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.384084    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c906b806-serving-cert\""
Oct 30 21:17:52.418203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.418167    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.418389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.418356    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ws2\" (UniqueName: \"kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.418897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.418870    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.419010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.418952    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.460215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.460150    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:52.520824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.520792    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.521046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.520942    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ws2\" (UniqueName: \"kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.523872 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:52.521888    4549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-serving-cert: secret "maas-api-e2e-derive-c906b806-serving-cert" not found
Oct 30 21:17:52.523872 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:52.521998    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls podName:8e67712e-96a9-42b9-9f9e-96da711f8492 nodeName:}" failed. No retries permitted until 2026-10-30 21:17:53.021958152 +0000 UTC m=+1055.796358090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls") pod "maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" (UID: "8e67712e-96a9-42b9-9f9e-96da711f8492") : secret "maas-api-e2e-derive-c906b806-serving-cert" not found
Oct 30 21:17:52.523872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.522269    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.523872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.522364    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.523872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.523455    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.536199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.524733    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.578866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.571886    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ws2\" (UniqueName: \"kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:52.591969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.590442    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:52.965964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:52.965927    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:53.039307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:53.039276    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:53.041989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:53.041956    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\") pod \"maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:53.271435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:53.271356    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:53.793155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:53.792894    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"]
Oct 30 21:17:53.793510 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:17:53.793208    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e67712e_96a9_42b9_9f9e_96da711f8492.slice/crio-be43e08dc895365337f7a487efc3d283dd2bd7fee96cb86fc5e784c3564dd9ff WatchSource:0}: Error finding container be43e08dc895365337f7a487efc3d283dd2bd7fee96cb86fc5e784c3564dd9ff: Status 404 returned error can't find the container with id be43e08dc895365337f7a487efc3d283dd2bd7fee96cb86fc5e784c3564dd9ff
Oct 30 21:17:53.991312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:53.991273    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" event={"ID":"8e67712e-96a9-42b9-9f9e-96da711f8492","Type":"ContainerStarted","Data":"be43e08dc895365337f7a487efc3d283dd2bd7fee96cb86fc5e784c3564dd9ff"}
Oct 30 21:17:54.105241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:54.105203    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:54.999442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:54.999401    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" event={"ID":"8e67712e-96a9-42b9-9f9e-96da711f8492","Type":"ContainerStarted","Data":"4281edb8e65254a946a5ebf82850f96b9d44bbc3a5a90f5e0aa0e919fd6a0373"}
Oct 30 21:17:54.999939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:54.999473    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:17:55.040671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:55.040611    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" podStartSLOduration=2.526679375 podStartE2EDuration="3.040594286s" totalImagesPullingTime="513.914911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:55.037388753 +0000 UTC m=+1057.811788691" watchObservedRunningTime="2026-10-30 21:17:55.040594286 +0000 UTC m=+1057.814994275"
Oct 30 21:17:55.667514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:55.667474    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:56.576081 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:17:56.576039    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:17:57.173164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:57.173126    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:57.989250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:57.989216    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:17:58.730850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:17:58.730789    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:00.227142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:00.227100    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:01.013281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:01.013247    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:18:01.763153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:01.763114    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:03.221983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:03.221949    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:04.636356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:04.636320    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:05.798333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:05.798290    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:06.059672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:06.059577    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.821084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:07.821051    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:08.239228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:08.239124    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:08.371033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:08.370988    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:09.405770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:09.405733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:10.996060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:10.996028    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:12.729866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:12.729804    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.429374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:14.429337    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.572092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:14.572062    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:15.846573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:15.846535    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:17.886076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:17.883314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:19.932571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:19.932528    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.322861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:21.321934    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:21.669913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:21.669562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.823860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:21.812299    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.966036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:21.965917    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:21.992659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:21.991435    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:22.424118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:22.424073    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:22.808379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:22.808297    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"]
Oct 30 21:18:23.010003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:23.009942    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:23.144755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:23.144720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:24.321084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:24.321045    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.612505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:25.612467    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.642676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:25.642642    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"]
Oct 30 21:18:25.642999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:25.642965    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58" podUID="17614e48-9d90-452d-b5eb-2dc001b7830c" containerName="maas-api" containerID="cri-o://855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936" gracePeriod=30
Oct 30 21:18:25.741640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:25.741596    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.929694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:25.929671    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:18:26.154560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.125700    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls\") pod \"17614e48-9d90-452d-b5eb-2dc001b7830c\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") "
Oct 30 21:18:26.155215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.155180    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca\") pod \"17614e48-9d90-452d-b5eb-2dc001b7830c\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") "
Oct 30 21:18:26.155495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.155471    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs\") pod \"17614e48-9d90-452d-b5eb-2dc001b7830c\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") "
Oct 30 21:18:26.155731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.155711    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq\" (UniqueName: \"kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq\") pod \"17614e48-9d90-452d-b5eb-2dc001b7830c\" (UID: \"17614e48-9d90-452d-b5eb-2dc001b7830c\") "
Oct 30 21:18:26.180037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.174404    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq" pod "17614e48-9d90-452d-b5eb-2dc001b7830c" (UID: "17614e48-9d90-452d-b5eb-2dc001b7830c"). InnerVolumeSpecName "kube-api-access-5glnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:26.189867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.187337    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca" pod "17614e48-9d90-452d-b5eb-2dc001b7830c" (UID: "17614e48-9d90-452d-b5eb-2dc001b7830c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:18:26.189867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.189057    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:26.204864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.203940    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs" pod "17614e48-9d90-452d-b5eb-2dc001b7830c" (UID: "17614e48-9d90-452d-b5eb-2dc001b7830c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:18:26.219856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.216308    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls" pod "17614e48-9d90-452d-b5eb-2dc001b7830c" (UID: "17614e48-9d90-452d-b5eb-2dc001b7830c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:18:26.266862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.261522    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:26.266862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.261555    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17614e48-9d90-452d-b5eb-2dc001b7830c-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:26.266862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.261585    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17614e48-9d90-452d-b5eb-2dc001b7830c-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:26.266862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.261600    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5glnq\" (UniqueName: \"kubernetes.io/projected/17614e48-9d90-452d-b5eb-2dc001b7830c-kube-api-access-5glnq\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:26.329855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.326853    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:26.635915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.635616    4549 generic.go:356] "Generic (PLEG): container finished" podID="17614e48-9d90-452d-b5eb-2dc001b7830c" containerID="855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936" exitCode=0
Oct 30 21:18:26.636434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.635953    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58" event={"ID":"17614e48-9d90-452d-b5eb-2dc001b7830c","Type":"ContainerDied","Data":"855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936"}
Oct 30 21:18:26.636434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.636023    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58" event={"ID":"17614e48-9d90-452d-b5eb-2dc001b7830c","Type":"ContainerDied","Data":"2ac09088e1cf9c299fd811077e143cbc2834c5d146956d1603b3605cc87092be"}
Oct 30 21:18:26.636434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.636056    4549 scope.go:122] "RemoveContainer" containerID="855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936"
Oct 30 21:18:26.636434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.636235    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"
Oct 30 21:18:26.746868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.742000    4549 scope.go:122] "RemoveContainer" containerID="855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936"
Oct 30 21:18:26.756135 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:18:26.755246    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936\": container with ID starting with 855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936 not found: ID does not exist" containerID="855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936"
Oct 30 21:18:26.756135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.755290    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936"} err="failed to get container status \"855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936\": rpc error: code = NotFound desc = could not find container \"855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936\": container with ID starting with 855c77b0634b4087444164b16644e94a42fc0995d6f3eba1e65a3f5159de7936 not found: ID does not exist"
Oct 30 21:18:26.810186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.810100    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"]
Oct 30 21:18:26.847859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:26.845888    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-399049f3-6bcc8c769b-dlc58"]
Oct 30 21:18:26.873823 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:18:26.873789    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:18:27.346922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.346867    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17614e48-9d90-452d-b5eb-2dc001b7830c" path="/var/lib/kubelet/pods/17614e48-9d90-452d-b5eb-2dc001b7830c/volumes"
Oct 30 21:18:27.808822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.808717    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h" podUID="e3373a9a-c86d-43b4-9fb7-18c075210666" containerName="payload-processing" containerID="cri-o://0b66ca25b80fb09dc76752fe9b7b205449eea35f46ff7f9e6a3a04dacdea5c37" gracePeriod=25
Oct 30 21:18:27.832987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.832961    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"]
Oct 30 21:18:27.833423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.833393    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" podUID="8e67712e-96a9-42b9-9f9e-96da711f8492" containerName="maas-api" containerID="cri-o://4281edb8e65254a946a5ebf82850f96b9d44bbc3a5a90f5e0aa0e919fd6a0373" gracePeriod=30
Oct 30 21:18:27.925321 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.925287    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:27.946565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.946538    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"]
Oct 30 21:18:27.947053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:27.947022    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" containerName="istio-proxy" containerID="cri-o://ebf2bb1770ad8aebd88708375190e1c6eba073b04b4011b7bae90abeea4c1a4f" gracePeriod=30
Oct 30 21:18:28.072914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:28.071875    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:28.910430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:28.910378    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.289909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.284900    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:29.350907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.347544    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.648857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.635992    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.649203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.645516    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"]
Oct 30 21:18:29.649593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.649560    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" podUID="46557a81-9192-41be-98e3-7f5043cfe6fe" containerName="istio-proxy" containerID="cri-o://1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9" gracePeriod=30
Oct 30 21:18:29.663617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.663543    4549 generic.go:356] "Generic (PLEG): container finished" podID="e3373a9a-c86d-43b4-9fb7-18c075210666" containerID="0b66ca25b80fb09dc76752fe9b7b205449eea35f46ff7f9e6a3a04dacdea5c37" exitCode=0
Oct 30 21:18:29.664101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.664038    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h" event={"ID":"e3373a9a-c86d-43b4-9fb7-18c075210666","Type":"ContainerDied","Data":"0b66ca25b80fb09dc76752fe9b7b205449eea35f46ff7f9e6a3a04dacdea5c37"}
Oct 30 21:18:29.664303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.664278    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h" event={"ID":"e3373a9a-c86d-43b4-9fb7-18c075210666","Type":"ContainerDied","Data":"06146a123a0a1730e515d8fca49ab32f2768b10b48dec09e6f6b2a6dee2b3422"}
Oct 30 21:18:29.664420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.664407    4549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06146a123a0a1730e515d8fca49ab32f2768b10b48dec09e6f6b2a6dee2b3422"
Oct 30 21:18:29.670864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.670816    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:18:29.677017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.676969    4549 generic.go:356] "Generic (PLEG): container finished" podID="8e67712e-96a9-42b9-9f9e-96da711f8492" containerID="4281edb8e65254a946a5ebf82850f96b9d44bbc3a5a90f5e0aa0e919fd6a0373" exitCode=0
Oct 30 21:18:29.680505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.677352    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:18:29.680753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.679893    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" event={"ID":"8e67712e-96a9-42b9-9f9e-96da711f8492","Type":"ContainerDied","Data":"4281edb8e65254a946a5ebf82850f96b9d44bbc3a5a90f5e0aa0e919fd6a0373"}
Oct 30 21:18:29.680971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.680920    4549 scope.go:122] "RemoveContainer" containerID="4281edb8e65254a946a5ebf82850f96b9d44bbc3a5a90f5e0aa0e919fd6a0373"
Oct 30 21:18:29.818897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.815190    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume\") pod \"e3373a9a-c86d-43b4-9fb7-18c075210666\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") "
Oct 30 21:18:29.818897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.815400    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\") pod \"8e67712e-96a9-42b9-9f9e-96da711f8492\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") "
Oct 30 21:18:29.818897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.815471    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs\") pod \"8e67712e-96a9-42b9-9f9e-96da711f8492\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") "
Oct 30 21:18:29.818897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.815636    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2\" (UniqueName: \"kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2\") pod \"8e67712e-96a9-42b9-9f9e-96da711f8492\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") "
Oct 30 21:18:29.818897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.815682    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca\") pod \"8e67712e-96a9-42b9-9f9e-96da711f8492\" (UID: \"8e67712e-96a9-42b9-9f9e-96da711f8492\") "
Oct 30 21:18:29.818897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.815776    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf\" (UniqueName: \"kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf\") pod \"e3373a9a-c86d-43b4-9fb7-18c075210666\" (UID: \"e3373a9a-c86d-43b4-9fb7-18c075210666\") "
Oct 30 21:18:29.872883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.832633    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca" pod "8e67712e-96a9-42b9-9f9e-96da711f8492" (UID: "8e67712e-96a9-42b9-9f9e-96da711f8492"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:18:29.872883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.835034    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume" pod "e3373a9a-c86d-43b4-9fb7-18c075210666" (UID: "e3373a9a-c86d-43b4-9fb7-18c075210666"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:18:29.872883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.860365    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs" pod "8e67712e-96a9-42b9-9f9e-96da711f8492" (UID: "8e67712e-96a9-42b9-9f9e-96da711f8492"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:18:29.872883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.862275    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2" pod "8e67712e-96a9-42b9-9f9e-96da711f8492" (UID: "8e67712e-96a9-42b9-9f9e-96da711f8492"). InnerVolumeSpecName "kube-api-access-q9ws2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:29.880858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.877043    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls" pod "8e67712e-96a9-42b9-9f9e-96da711f8492" (UID: "8e67712e-96a9-42b9-9f9e-96da711f8492"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:18:29.916116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.886700    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf" pod "e3373a9a-c86d-43b4-9fb7-18c075210666" (UID: "e3373a9a-c86d-43b4-9fb7-18c075210666"). InnerVolumeSpecName "kube-api-access-5jrnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:29.916116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.913552    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.939435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.922177    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:29.939435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.922229    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e67712e-96a9-42b9-9f9e-96da711f8492-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:29.939435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.922246    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9ws2\" (UniqueName: \"kubernetes.io/projected/8e67712e-96a9-42b9-9f9e-96da711f8492-kube-api-access-q9ws2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:29.939435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.922259    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e67712e-96a9-42b9-9f9e-96da711f8492-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:29.939435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.922273    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jrnf\" (UniqueName: \"kubernetes.io/projected/e3373a9a-c86d-43b4-9fb7-18c075210666-kube-api-access-5jrnf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:29.939435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:29.922287    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3373a9a-c86d-43b4-9fb7-18c075210666-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:30.233465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.233379    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"]
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234212    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e3373a9a-c86d-43b4-9fb7-18c075210666" containerName="payload-processing"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234229    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3373a9a-c86d-43b4-9fb7-18c075210666" containerName="payload-processing"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234246    4549 cpu_manager.go:414] "containerMap: removing container" podUID="17614e48-9d90-452d-b5eb-2dc001b7830c" containerName="maas-api"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234251    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17614e48-9d90-452d-b5eb-2dc001b7830c" containerName="maas-api"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234266    4549 cpu_manager.go:414] "containerMap: removing container" podUID="8e67712e-96a9-42b9-9f9e-96da711f8492" containerName="maas-api"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234271    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e67712e-96a9-42b9-9f9e-96da711f8492" containerName="maas-api"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234423    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3373a9a-c86d-43b4-9fb7-18c075210666" containerName="payload-processing"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234437    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="17614e48-9d90-452d-b5eb-2dc001b7830c" containerName="maas-api"
Oct 30 21:18:30.236854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.234449    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e67712e-96a9-42b9-9f9e-96da711f8492" containerName="maas-api"
Oct 30 21:18:30.240911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.237693    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.261342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.261307    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-e11bef-openshift-default-dockercfg-7rbl5\""
Oct 30 21:18:30.282367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.282328    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"]
Oct 30 21:18:30.328488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328450    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.328680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328512    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.328680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328544    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.328680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328596    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.328680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328673    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.328939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328701    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7lsn\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.328939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.328905    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.329046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.329013    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.329096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.329073    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.339514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.339481    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451040    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451189    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451258    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451313    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451356    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451389    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451443    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451533    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.451565    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7lsn\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.455807    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.459861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.456547    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.476855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.469953    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.476855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.470192    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.476855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.470494    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.476855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.476682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:30.498907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.493020    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.509872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.508499    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7lsn\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.522853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.518655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.563858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.563752    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy\") pod \"e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") " pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.720867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.719547    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"
Oct 30 21:18:30.738856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.733141    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"
Oct 30 21:18:30.757910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.754959    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh" event={"ID":"8e67712e-96a9-42b9-9f9e-96da711f8492","Type":"ContainerDied","Data":"be43e08dc895365337f7a487efc3d283dd2bd7fee96cb86fc5e784c3564dd9ff"}
Oct 30 21:18:30.861811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.861772    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:30.999639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:30.999235    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"]
Oct 30 21:18:31.032969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.032870    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h"]
Oct 30 21:18:31.060253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.059731    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"]
Oct 30 21:18:31.066923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.066789    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c906b806-c4c4fd5fd-sp7nh"]
Oct 30 21:18:31.348953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.348916    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e67712e-96a9-42b9-9f9e-96da711f8492" path="/var/lib/kubelet/pods/8e67712e-96a9-42b9-9f9e-96da711f8492/volumes"
Oct 30 21:18:31.349421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.349406    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3373a9a-c86d-43b4-9fb7-18c075210666" path="/var/lib/kubelet/pods/e3373a9a-c86d-43b4-9fb7-18c075210666/volumes"
Oct 30 21:18:31.400306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.400270    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"]
Oct 30 21:18:31.402053 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:18:31.402027    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb817463_80b7_427d_b20f_5a1c6914f87d.slice/crio-a9bf6ddd131f5a51a347ede326f23b86f17a441e5688fa019400f53ad666c79f WatchSource:0}: Error finding container a9bf6ddd131f5a51a347ede326f23b86f17a441e5688fa019400f53ad666c79f: Status 404 returned error can't find the container with id a9bf6ddd131f5a51a347ede326f23b86f17a441e5688fa019400f53ad666c79f
Oct 30 21:18:31.405196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.405162    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:18:31.405265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.405229    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:18:31.405265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.405258    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:18:31.731604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.731516    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" event={"ID":"bb817463-80b7-427d-b20f-5a1c6914f87d","Type":"ContainerStarted","Data":"01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"}
Oct 30 21:18:31.731910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.731600    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" event={"ID":"bb817463-80b7-427d-b20f-5a1c6914f87d","Type":"ContainerStarted","Data":"a9bf6ddd131f5a51a347ede326f23b86f17a441e5688fa019400f53ad666c79f"}
Oct 30 21:18:31.804676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.804602    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podStartSLOduration=1.804580418 podStartE2EDuration="1.804580418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:31.792436677 +0000 UTC m=+1094.566836619" watchObservedRunningTime="2026-10-30 21:18:31.804580418 +0000 UTC m=+1094.578980357"
Oct 30 21:18:31.863553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.863521    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:31.948099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:31.948067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:32.444016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:32.443972    4549 patch_prober.go:29] interesting pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:18:32.444511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:32.444041    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:32.873857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:32.868847    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:33.565099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.564742    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:33.763917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.763807    4549 generic.go:356] "Generic (PLEG): container finished" podID="5a23f05c-f215-4618-9e40-1bab6a597276" containerID="ebf2bb1770ad8aebd88708375190e1c6eba073b04b4011b7bae90abeea4c1a4f" exitCode=0
Oct 30 21:18:33.764662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.764147    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" event={"ID":"5a23f05c-f215-4618-9e40-1bab6a597276","Type":"ContainerDied","Data":"ebf2bb1770ad8aebd88708375190e1c6eba073b04b4011b7bae90abeea4c1a4f"}
Oct 30 21:18:33.766725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.765149    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:33.768467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.768208    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:18:33.966155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.966124    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:18:33.992333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992304    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.992497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992380    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.992497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992401    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.992497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992483    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.992624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992574    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.992692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992650    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.992780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992739    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:33.992780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992765    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.993067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992727    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:33.993163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992803    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.993163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.993077    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:18:33.993163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.992998    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:33.993494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.993228    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs\") pod \"5a23f05c-f215-4618-9e40-1bab6a597276\" (UID: \"5a23f05c-f215-4618-9e40-1bab6a597276\") "
Oct 30 21:18:33.993711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.993654    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:33.994230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.994204    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:33.994336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.994234    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:33.994336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.994248    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:33.994336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.994263    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a23f05c-f215-4618-9e40-1bab6a597276-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:33.994336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.994276    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:33.995255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.995232    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:33.995384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.995359    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "kube-api-access-p48b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:33.995548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.995532    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:18:33.995647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:33.995624    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy" pod "5a23f05c-f215-4618-9e40-1bab6a597276" (UID: "5a23f05c-f215-4618-9e40-1bab6a597276"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:34.096398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.096362    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a23f05c-f215-4618-9e40-1bab6a597276-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:34.096398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.096393    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:34.096398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.096402    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a23f05c-f215-4618-9e40-1bab6a597276-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:34.096642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.096411    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p48b5\" (UniqueName: \"kubernetes.io/projected/5a23f05c-f215-4618-9e40-1bab6a597276-kube-api-access-p48b5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:34.261991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.261902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:34.497007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.496964    4549 patch_prober.go:29] interesting pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:18:34.497201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.497057    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" podUID="46557a81-9192-41be-98e3-7f5043cfe6fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:34.541674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.541583    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:34.778879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.776085    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"
Oct 30 21:18:34.778879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.776113    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv" event={"ID":"5a23f05c-f215-4618-9e40-1bab6a597276","Type":"ContainerDied","Data":"250648f05824ded3a7a156ceba520a398594bcfb827cf52f9fc65746f6b74d5b"}
Oct 30 21:18:34.778879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.776185    4549 scope.go:122] "RemoveContainer" containerID="ebf2bb1770ad8aebd88708375190e1c6eba073b04b4011b7bae90abeea4c1a4f"
Oct 30 21:18:34.832581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.832533    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"]
Oct 30 21:18:34.844569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:34.844532    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv"]
Oct 30 21:18:35.290693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:35.290594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.401705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:35.397234    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" path="/var/lib/kubelet/pods/5a23f05c-f215-4618-9e40-1bab6a597276/volumes"
Oct 30 21:18:35.949608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:35.949571    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.100139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.100103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.102510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.102479    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"]
Oct 30 21:18:36.103257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.103237    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" containerName="istio-proxy"
Oct 30 21:18:36.103257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.103256    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" containerName="istio-proxy"
Oct 30 21:18:36.103417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.103388    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a23f05c-f215-4618-9e40-1bab6a597276" containerName="istio-proxy"
Oct 30 21:18:36.108253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.108236    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.132019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.131987    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"]
Oct 30 21:18:36.138070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.137639    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-e11bef\""
Oct 30 21:18:36.138070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.137748    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-e11bef-dockercfg-7dvwl\""
Oct 30 21:18:36.254071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.253974    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4bvh\" (UniqueName: \"kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh\") pod \"payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.254451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.254422    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.388864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.380881    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4bvh\" (UniqueName: \"kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh\") pod \"payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.388864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.381081    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.388864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.382255    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.564472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.564363    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"]
Oct 30 21:18:36.592235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.592070    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"]
Oct 30 21:18:36.611963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.598058    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.630233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.621687    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4bvh\" (UniqueName: \"kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh\") pod \"payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.630233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.626606    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.630233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.626706    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mff68\" (UniqueName: \"kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68\") pod \"payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.724016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.723981    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:36.728748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.728726    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.728841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.728811    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff68\" (UniqueName: \"kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68\") pod \"payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.729556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.729536    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.755451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.755423    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mff68\" (UniqueName: \"kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68\") pod \"payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:36.948465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:36.948432    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:37.043724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.043689    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"]
Oct 30 21:18:37.052263 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:18:37.052228    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffda1237_0394_47d7_bcbb_fddae855ef6d.slice/crio-48d14d12e8000846483e8ce7b53d9ccd75b2bee845beb0f7eed50ef5e775f708 WatchSource:0}: Error finding container 48d14d12e8000846483e8ce7b53d9ccd75b2bee845beb0f7eed50ef5e775f708: Status 404 returned error can't find the container with id 48d14d12e8000846483e8ce7b53d9ccd75b2bee845beb0f7eed50ef5e775f708
Oct 30 21:18:37.154574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.154522    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.328915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.305298    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"]
Oct 30 21:18:37.329428 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:18:37.317574    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb526a0b0_b173_4352_8c9e_e868c1416071.slice/crio-7021223440d9782c0802f730cd5ac3e360502401fefbd0bf3de4e22d24de910e WatchSource:0}: Error finding container 7021223440d9782c0802f730cd5ac3e360502401fefbd0bf3de4e22d24de910e: Status 404 returned error can't find the container with id 7021223440d9782c0802f730cd5ac3e360502401fefbd0bf3de4e22d24de910e
Oct 30 21:18:37.813443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.813334    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" event={"ID":"b526a0b0-b173-4352-8c9e-e868c1416071","Type":"ContainerStarted","Data":"e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173"}
Oct 30 21:18:37.813443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.813406    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" event={"ID":"b526a0b0-b173-4352-8c9e-e868c1416071","Type":"ContainerStarted","Data":"7021223440d9782c0802f730cd5ac3e360502401fefbd0bf3de4e22d24de910e"}
Oct 30 21:18:37.813443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.813435    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:37.815197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.815167    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" event={"ID":"ffda1237-0394-47d7-bcbb-fddae855ef6d","Type":"ContainerStarted","Data":"02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227"}
Oct 30 21:18:37.815313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.815216    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" event={"ID":"ffda1237-0394-47d7-bcbb-fddae855ef6d","Type":"ContainerStarted","Data":"48d14d12e8000846483e8ce7b53d9ccd75b2bee845beb0f7eed50ef5e775f708"}
Oct 30 21:18:37.815313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.815304    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:37.856597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.856551    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" podStartSLOduration=1.856537743 podStartE2EDuration="1.856537743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:37.852315167 +0000 UTC m=+1100.626715107" watchObservedRunningTime="2026-10-30 21:18:37.856537743 +0000 UTC m=+1100.630937703"
Oct 30 21:18:37.894542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:37.894482    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" podStartSLOduration=1.894471199 podStartE2EDuration="1.894471199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:37.891090889 +0000 UTC m=+1100.665490831" watchObservedRunningTime="2026-10-30 21:18:37.894471199 +0000 UTC m=+1100.668871135"
Oct 30 21:18:38.045927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.044446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:38.443077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.443051    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:18:38.476679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.476647    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.476679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.476687    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.476961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.476810    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.476961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.476891    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.476961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.476939    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.477118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.476977    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.477118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477101    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.477217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477134    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.477274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477095    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:38.477274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477221    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token\") pod \"46557a81-9192-41be-98e3-7f5043cfe6fe\" (UID: \"46557a81-9192-41be-98e3-7f5043cfe6fe\") "
Oct 30 21:18:38.477274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477119    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:18:38.477274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477240    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:38.477523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477289    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:38.477620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.477599    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:38.478242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.478217    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.478242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.478241    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.478439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.478254    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.478439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.478262    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46557a81-9192-41be-98e3-7f5043cfe6fe-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.478439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.478271    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.479787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.479758    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:18:38.480074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.480045    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:38.480170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.480075    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:18:38.480210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.480177    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt" pod "46557a81-9192-41be-98e3-7f5043cfe6fe" (UID: "46557a81-9192-41be-98e3-7f5043cfe6fe"). InnerVolumeSpecName "kube-api-access-2t7gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:38.589197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.589153    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t7gt\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-kube-api-access-2t7gt\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.589197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.589196    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.589438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.589213    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.589438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.589225    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46557a81-9192-41be-98e3-7f5043cfe6fe-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:18:38.823203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.823173    4549 generic.go:356] "Generic (PLEG): container finished" podID="46557a81-9192-41be-98e3-7f5043cfe6fe" containerID="1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9" exitCode=0
Oct 30 21:18:38.823388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.823245    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"
Oct 30 21:18:38.823388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.823263    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" event={"ID":"46557a81-9192-41be-98e3-7f5043cfe6fe","Type":"ContainerDied","Data":"1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9"}
Oct 30 21:18:38.823388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.823358    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n" event={"ID":"46557a81-9192-41be-98e3-7f5043cfe6fe","Type":"ContainerDied","Data":"eee8e0b54d6c7f791556e2472ec378c2197f5c56c88eefc63d5d3ee43d8f4452"}
Oct 30 21:18:38.823388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.823383    4549 scope.go:122] "RemoveContainer" containerID="1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9"
Oct 30 21:18:38.833595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.833569    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:38.835430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.835410    4549 scope.go:122] "RemoveContainer" containerID="1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9"
Oct 30 21:18:38.835682 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:18:38.835660    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9\": container with ID starting with 1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9 not found: ID does not exist" containerID="1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9"
Oct 30 21:18:38.835744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.835690    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9"} err="failed to get container status \"1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9\": rpc error: code = NotFound desc = could not find container \"1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9\": container with ID starting with 1e933124eb3f2fdf203a1bcacd61070898ec1f698acf5329da74a4696989d8b9 not found: ID does not exist"
Oct 30 21:18:38.907954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.907886    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"]
Oct 30 21:18:38.914246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:38.914208    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n"]
Oct 30 21:18:39.345549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:39.345515    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46557a81-9192-41be-98e3-7f5043cfe6fe" path="/var/lib/kubelet/pods/46557a81-9192-41be-98e3-7f5043cfe6fe/volumes"
Oct 30 21:18:40.408381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:40.408336    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:41.863025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:41.862993    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:43.435392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:43.435345    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.259291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:44.259253    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.791258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:44.791227    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:46.182095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:46.182064    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.500425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.500385    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"]
Oct 30 21:18:47.501337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.501313    4549 cpu_manager.go:414] "containerMap: removing container" podUID="46557a81-9192-41be-98e3-7f5043cfe6fe" containerName="istio-proxy"
Oct 30 21:18:47.501443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.501338    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46557a81-9192-41be-98e3-7f5043cfe6fe" containerName="istio-proxy"
Oct 30 21:18:47.501520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.501506    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="46557a81-9192-41be-98e3-7f5043cfe6fe" containerName="istio-proxy"
Oct 30 21:18:47.504911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.504891    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.523436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.521157    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-e11bef-serving-cert\""
Oct 30 21:18:47.528097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.528049    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"]
Oct 30 21:18:47.575381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.575340    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.575588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.575396    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htqgm\" (UniqueName: \"kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.575666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.575628    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.575874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.575815    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.596393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.596353    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.677742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.677694    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.677976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.677818    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.677976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.677916    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.677976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.677934    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqgm\" (UniqueName: \"kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.678407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.678384    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.680618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.680595    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.680714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.680639    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.697625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.697585    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqgm\" (UniqueName: \"kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm\") pod \"maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:47.750922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.740386    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.838858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:47.838228    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:48.229206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:48.229175    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"]
Oct 30 21:18:48.229682 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:18:48.229658    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b350545_9354_45da_9481_8012e1f8d398.slice/crio-4c1023c11d0b8bfcef5f2b502ea5910eeeb0d77d43e2162e5e831917f1f0d421 WatchSource:0}: Error finding container 4c1023c11d0b8bfcef5f2b502ea5910eeeb0d77d43e2162e5e831917f1f0d421: Status 404 returned error can't find the container with id 4c1023c11d0b8bfcef5f2b502ea5910eeeb0d77d43e2162e5e831917f1f0d421
Oct 30 21:18:48.832096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:48.832069    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:18:48.833329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:48.833299    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:18:48.955918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:48.955874    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" event={"ID":"8b350545-9354-45da-9481-8012e1f8d398","Type":"ContainerStarted","Data":"5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611"}
Oct 30 21:18:48.956154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:48.955955    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:48.956154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:48.955984    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" event={"ID":"8b350545-9354-45da-9481-8012e1f8d398","Type":"ContainerStarted","Data":"4c1023c11d0b8bfcef5f2b502ea5910eeeb0d77d43e2162e5e831917f1f0d421"}
Oct 30 21:18:49.003886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:49.003789    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" podStartSLOduration=1.694856266 podStartE2EDuration="2.00377361s" totalImagesPullingTime="308.917344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:18:49.000207092 +0000 UTC m=+1111.774607082" watchObservedRunningTime="2026-10-30 21:18:49.00377361 +0000 UTC m=+1111.778173548"
Oct 30 21:18:49.464096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:49.464060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:51.006294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:51.006256    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:52.497099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:52.497056    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:52.621740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:52.621624    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:54.066859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:54.065494    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:54.971133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:54.971100    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:18:55.497484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:55.497452    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.577230 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:18:56.577187    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:18:56.876469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:56.876383    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:58.343739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:58.343707    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:58.987681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:58.987647    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:59.770433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:18:59.770402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:01.145878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:01.145809    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:02.861131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:02.860980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:04.518530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:04.518492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:05.903745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:05.903712    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:06.101543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:06.101506    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.536460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:07.536411    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:09.647975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:09.647908    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:09.780507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:09.780448    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.214175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:10.214104    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.694857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:10.654517    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"]
Oct 30 21:19:10.945909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:10.919095    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.042893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.042857    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"]
Oct 30 21:19:11.043244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.043226    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.094902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.092197    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-c46397-openshift-default-dockercfg-pbp7k\""
Oct 30 21:19:11.096780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.096751    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143416    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143541    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143593    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143749    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143786    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143844    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143880    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143919    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.167858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.143940    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vswz\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.247713    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.247799    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.247989    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.248028    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.248075    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.248113    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.248152    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.248176    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vswz\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.298160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.248299    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.375952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.328097    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.375952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.328373    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.413987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.408780    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.481502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.447228    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.481502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.472446    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.564322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.564232    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.598851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.597110    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.695880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.674906    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vswz\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.695880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.687001    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token\") pod \"e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") " pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.783016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.782963    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:11.904856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.899212    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:19:11.905416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.905385    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:11.971085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.964284    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:11.974940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:11.974878    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:12.144379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.144337    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:19:12.146049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.145702    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:12.331957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.331910    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:12.515806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.515768    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:19:12.516153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.516124    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:12.813030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.812918    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:19:12.858350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.858120    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:19:12.992364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.967879    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"]
Oct 30 21:19:12.992364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.965121    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:12.992364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.971915    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:19:12.992364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.975936    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:12.992364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:12.976413    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:19:13.089447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.054667    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:13.089447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.054765    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:13.089447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.054821    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:13.168873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.157233    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:19:13.168873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.158220    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:19:13.310206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.299758    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:19:13.743254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.743154    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:19:13.745897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.745872    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:19:13.769883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.769454    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:19:13.769883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.769520    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:19:13.858107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:13.856700    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" event={"ID":"71f46951-9378-4dee-9f2e-42d72da5ee1f","Type":"ContainerStarted","Data":"d09577ee5359e6c5f6c6333d238c66ae9af619bf60d0652c3b9ca9b631d08b9c"}
Oct 30 21:19:14.444192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:14.444158    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:14.864236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:14.864192    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" event={"ID":"71f46951-9378-4dee-9f2e-42d72da5ee1f","Type":"ContainerStarted","Data":"e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"}
Oct 30 21:19:14.910317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:14.910281    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:14.916402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:14.916376    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:14.946186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:14.946136    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podStartSLOduration=4.946123798 podStartE2EDuration="4.946123798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:14.9315295 +0000 UTC m=+1137.705929621" watchObservedRunningTime="2026-10-30 21:19:14.946123798 +0000 UTC m=+1137.720523736"
Oct 30 21:19:15.897300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:15.897090    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:15.898534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:15.898496    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:19:16.525777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:16.525740    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:17.330271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:17.330211    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:17.459095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:17.459040    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:18.207712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:18.207348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:18.980664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:18.980626    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.108536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.108473    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.124804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.124768    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"]
Oct 30 21:19:19.131943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.131919    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.151484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.151457    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"]
Oct 30 21:19:19.158262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.158237    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-c46397\""
Oct 30 21:19:19.232956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.204858    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6ghq\" (UniqueName: \"kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq\") pod \"payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.232956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.205160    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.293950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.293914    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-c46397-dockercfg-zh2zm\""
Oct 30 21:19:19.308957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.308921    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.309440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.309416    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6ghq\" (UniqueName: \"kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq\") pod \"payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.310994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.310970    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.326975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.326940    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"]
Oct 30 21:19:19.461055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.461015    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.542865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.542505    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.542865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.542671    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk4kn\" (UniqueName: \"kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn\") pod \"payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.579269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.579234    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"]
Oct 30 21:19:19.585866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.585794    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6ghq\" (UniqueName: \"kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq\") pod \"payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.645584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.644485    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.645893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.645533    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.646207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.646187    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk4kn\" (UniqueName: \"kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn\") pod \"payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.676196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.668286    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk4kn\" (UniqueName: \"kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn\") pod \"payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:19.747945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.747819    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:19.807379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.806991    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:19:19.807379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.807072    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:19.888460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:19.888420    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:20.019152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.019061    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:20.115036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.114986    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"]
Oct 30 21:19:20.239968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.239939    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"]
Oct 30 21:19:20.946650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.946608    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" event={"ID":"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e","Type":"ContainerStarted","Data":"072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632"}
Oct 30 21:19:20.946991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.946954    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:20.947110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.947019    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" event={"ID":"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e","Type":"ContainerStarted","Data":"78e11666ce0c7798b846a85e41b0e881b2f4667b1352e9636a58c652e30d3683"}
Oct 30 21:19:20.948334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.948312    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" event={"ID":"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d","Type":"ContainerStarted","Data":"eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7"}
Oct 30 21:19:20.948440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.948348    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" event={"ID":"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d","Type":"ContainerStarted","Data":"5a7175c368a178c3822831f6fc7af737e315043e8166f1129bbc50c53bdf260e"}
Oct 30 21:19:20.948551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.948530    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:20.984389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:20.984319    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" podStartSLOduration=1.98430158 podStartE2EDuration="1.98430158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:20.983907546 +0000 UTC m=+1143.758307488" watchObservedRunningTime="2026-10-30 21:19:20.98430158 +0000 UTC m=+1143.758701520"
Oct 30 21:19:21.025043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.024985    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" podStartSLOduration=2.024970552 podStartE2EDuration="2.024970552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:21.020406359 +0000 UTC m=+1143.794806300" watchObservedRunningTime="2026-10-30 21:19:21.024970552 +0000 UTC m=+1143.799370491"
Oct 30 21:19:21.697857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.697808    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:21.957260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.957172    4549 generic.go:356] "Generic (PLEG): container finished" podID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerID="01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157" exitCode=0
Oct 30 21:19:21.957260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.957248    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" event={"ID":"bb817463-80b7-427d-b20f-5a1c6914f87d","Type":"ContainerDied","Data":"01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"}
Oct 30 21:19:21.957777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.957761    4549 scope.go:122] "RemoveContainer" containerID="01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"
Oct 30 21:19:21.959278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.959248    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:21.959379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.959313    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:21.959379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:21.959346    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:22.966873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:22.966813    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" event={"ID":"bb817463-80b7-427d-b20f-5a1c6914f87d","Type":"ContainerStarted","Data":"6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93"}
Oct 30 21:19:23.213271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:23.213242    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:23.862733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:23.862701    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:19:23.862951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:23.862899    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:19:23.863013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:23.862943    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 21:19:24.598175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:24.598137    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"]
Oct 30 21:19:24.650746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:24.650705    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"]
Oct 30 21:19:24.767342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:24.767307    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:24.904081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:24.903986    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:19:24.904491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:24.904452    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 21:19:24.936312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:24.936166    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:25.862956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:25.862922    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:19:25.863384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:25.862985    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 21:19:26.602103 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:19:26.602066    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:19:26.618325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:26.618292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:26.863139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:26.863051    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:19:26.863139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:26.863101    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 21:19:27.862895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:27.862858    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:19:27.863077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:27.862926    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 21:19:28.098890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:28.098821    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:28.863595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:28.863553    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:19:28.863795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:28.863627    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 21:19:29.598909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:29.598812    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w" podUID="82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" containerName="payload-processing" containerID="cri-o://024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9" gracePeriod=25
Oct 30 21:19:29.651865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:29.651754    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx" podUID="1476163b-023f-43b3-8947-f778ba1aaed0" containerName="payload-pre-processing" containerID="cri-o://d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5" gracePeriod=25
Oct 30 21:19:29.869392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:29.869308    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:19:29.869645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:29.869620    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:19:29.870576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:29.870554    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:19:30.135206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.125589    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.142233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.142202    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:19:30.208192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.208160    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:19:30.255130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.255063    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume\") pod \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") "
Oct 30 21:19:30.255130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.255134    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume\") pod \"1476163b-023f-43b3-8947-f778ba1aaed0\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") "
Oct 30 21:19:30.255355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.255221    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r\" (UniqueName: \"kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r\") pod \"1476163b-023f-43b3-8947-f778ba1aaed0\" (UID: \"1476163b-023f-43b3-8947-f778ba1aaed0\") "
Oct 30 21:19:30.255884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.255403    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8\" (UniqueName: \"kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8\") pod \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\" (UID: \"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8\") "
Oct 30 21:19:30.255884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.255556    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume" pod "82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" (UID: "82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:19:30.256060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.256025    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume" pod "1476163b-023f-43b3-8947-f778ba1aaed0" (UID: "1476163b-023f-43b3-8947-f778ba1aaed0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:19:30.256118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.256076    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:30.259004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.258979    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8" pod "82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" (UID: "82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8"). InnerVolumeSpecName "kube-api-access-s4xp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:19:30.259178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.259162    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r" pod "1476163b-023f-43b3-8947-f778ba1aaed0" (UID: "1476163b-023f-43b3-8947-f778ba1aaed0"). InnerVolumeSpecName "kube-api-access-rpt4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:19:30.305418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.305370    4549 generic.go:356] "Generic (PLEG): container finished" podID="82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" containerID="024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9" exitCode=0
Oct 30 21:19:30.305598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.305503    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"
Oct 30 21:19:30.305598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.305519    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w" event={"ID":"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8","Type":"ContainerDied","Data":"024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9"}
Oct 30 21:19:30.305598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.305583    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w" event={"ID":"82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8","Type":"ContainerDied","Data":"ab02cd0d09b54f6605a743e34d88a7e2b01faa4e6a2e2c3ac51d8da7a602cf87"}
Oct 30 21:19:30.305851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.305617    4549 scope.go:122] "RemoveContainer" containerID="024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9"
Oct 30 21:19:30.307322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.307300    4549 generic.go:356] "Generic (PLEG): container finished" podID="1476163b-023f-43b3-8947-f778ba1aaed0" containerID="d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5" exitCode=0
Oct 30 21:19:30.307448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.307357    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx" event={"ID":"1476163b-023f-43b3-8947-f778ba1aaed0","Type":"ContainerDied","Data":"d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5"}
Oct 30 21:19:30.307448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.307408    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx" event={"ID":"1476163b-023f-43b3-8947-f778ba1aaed0","Type":"ContainerDied","Data":"6312a6147379a9a14072cd01990749118fe5c5c00327c06b74ef159c3fac7aa5"}
Oct 30 21:19:30.318581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.318563    4549 scope.go:122] "RemoveContainer" containerID="024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9"
Oct 30 21:19:30.318922 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:19:30.318903    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9\": container with ID starting with 024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9 not found: ID does not exist" containerID="024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9"
Oct 30 21:19:30.319004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.318934    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9"} err="failed to get container status \"024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9\": rpc error: code = NotFound desc = could not find container \"024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9\": container with ID starting with 024dcae38e4e87d94e301ea2f57146185b4a34622d602aa030728a399f6baea9 not found: ID does not exist"
Oct 30 21:19:30.319004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.318961    4549 scope.go:122] "RemoveContainer" containerID="d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5"
Oct 30 21:19:30.319127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.319070    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"
Oct 30 21:19:30.330319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.330300    4549 scope.go:122] "RemoveContainer" containerID="d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5"
Oct 30 21:19:30.330611 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:19:30.330589    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5\": container with ID starting with d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5 not found: ID does not exist" containerID="d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5"
Oct 30 21:19:30.330675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.330624    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5"} err="failed to get container status \"d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5\": rpc error: code = NotFound desc = could not find container \"d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5\": container with ID starting with d7d225ab6ba01455b03fc585410bc843c4fcba92d09ec13ca0d917738d3356a5 not found: ID does not exist"
Oct 30 21:19:30.359045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.359016    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpt4r\" (UniqueName: \"kubernetes.io/projected/1476163b-023f-43b3-8947-f778ba1aaed0-kube-api-access-rpt4r\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:30.359045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.359048    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4xp8\" (UniqueName: \"kubernetes.io/projected/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8-kube-api-access-s4xp8\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:30.359245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.359064    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1476163b-023f-43b3-8947-f778ba1aaed0-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:30.370079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.370043    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"]
Oct 30 21:19:30.373770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.373705    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w"]
Oct 30 21:19:30.406324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.406295    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"]
Oct 30 21:19:30.411895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:30.411865    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx"]
Oct 30 21:19:31.345311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:31.345272    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1476163b-023f-43b3-8947-f778ba1aaed0" path="/var/lib/kubelet/pods/1476163b-023f-43b3-8947-f778ba1aaed0/volumes"
Oct 30 21:19:31.346131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:31.346110    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" path="/var/lib/kubelet/pods/82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8/volumes"
Oct 30 21:19:31.680813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:31.680720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:31.960966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:31.960881    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:19:31.960966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:31.960946    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:19:33.267200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:33.267161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:33.399683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:33.399646    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:33.535914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:33.534055    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:35.069305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:35.069270    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:36.564205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:36.564169    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:37.773368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:37.773318    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"]
Oct 30 21:19:37.774600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:37.774535    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" podUID="2a74a760-d184-4219-a631-7a07cb625c19" containerName="maas-api" containerID="cri-o://7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92" gracePeriod=30
Oct 30 21:19:37.858246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:37.858216    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.436856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.419591    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.436856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.436754    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:19:38.531905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.520698    4549 generic.go:356] "Generic (PLEG): container finished" podID="2a74a760-d184-4219-a631-7a07cb625c19" containerID="7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92" exitCode=0
Oct 30 21:19:38.531905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.520921    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" event={"ID":"2a74a760-d184-4219-a631-7a07cb625c19","Type":"ContainerDied","Data":"7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92"}
Oct 30 21:19:38.531905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.520990    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n" event={"ID":"2a74a760-d184-4219-a631-7a07cb625c19","Type":"ContainerDied","Data":"256e501bd3d0c43061eaca4f7383cbb7c433c1a7e3174582b24be29258f7d0f0"}
Oct 30 21:19:38.531905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.521024    4549 scope.go:122] "RemoveContainer" containerID="7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92"
Oct 30 21:19:38.531905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.521170    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"
Oct 30 21:19:38.537082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.537051    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.640472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.620314    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs\") pod \"2a74a760-d184-4219-a631-7a07cb625c19\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") "
Oct 30 21:19:38.640472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.620436    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\") pod \"2a74a760-d184-4219-a631-7a07cb625c19\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") "
Oct 30 21:19:38.640472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.620637    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca\") pod \"2a74a760-d184-4219-a631-7a07cb625c19\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") "
Oct 30 21:19:38.640472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.620737    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj\" (UniqueName: \"kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj\") pod \"2a74a760-d184-4219-a631-7a07cb625c19\" (UID: \"2a74a760-d184-4219-a631-7a07cb625c19\") "
Oct 30 21:19:38.671855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.663058    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca" pod "2a74a760-d184-4219-a631-7a07cb625c19" (UID: "2a74a760-d184-4219-a631-7a07cb625c19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:19:38.681983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.681295    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj" pod "2a74a760-d184-4219-a631-7a07cb625c19" (UID: "2a74a760-d184-4219-a631-7a07cb625c19"). InnerVolumeSpecName "kube-api-access-j5zrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:19:38.682514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.682457    4549 scope.go:122] "RemoveContainer" containerID="7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92"
Oct 30 21:19:38.712510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.712444    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5zrj\" (UniqueName: \"kubernetes.io/projected/2a74a760-d184-4219-a631-7a07cb625c19-kube-api-access-j5zrj\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:38.712510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.712479    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a74a760-d184-4219-a631-7a07cb625c19-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:38.713670 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:19:38.713144    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92\": container with ID starting with 7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92 not found: ID does not exist" containerID="7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92"
Oct 30 21:19:38.713670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.713188    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92"} err="failed to get container status \"7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92\": rpc error: code = NotFound desc = could not find container \"7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92\": container with ID starting with 7e7ff8781a5aa98f3687076015989c19ea6e2055b07028354943c3aa29987f92 not found: ID does not exist"
Oct 30 21:19:38.713670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.713587    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.765271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.765230    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls" pod "2a74a760-d184-4219-a631-7a07cb625c19" (UID: "2a74a760-d184-4219-a631-7a07cb625c19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:38.782579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.781876    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs" pod "2a74a760-d184-4219-a631-7a07cb625c19" (UID: "2a74a760-d184-4219-a631-7a07cb625c19"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:38.814474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.814415    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:38.814731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.814714    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a74a760-d184-4219-a631-7a07cb625c19-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:19:38.957604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.957570    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"]
Oct 30 21:19:38.968237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:38.968205    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f954ca9-fc6fbdc44-gpl6n"]
Oct 30 21:19:39.164906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:39.164865    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:39.346319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:39.346287    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a74a760-d184-4219-a631-7a07cb625c19" path="/var/lib/kubelet/pods/2a74a760-d184-4219-a631-7a07cb625c19/volumes"
Oct 30 21:19:40.099260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:40.099223    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:40.516547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:40.516461    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:42.145854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:42.140198    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:43.691304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:43.691272    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.234253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:45.234218    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.423819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:45.423613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.683946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:45.683916    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:46.156554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:46.156517    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:46.292717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:46.292681    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:46.474630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:46.474539    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:47.074220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:47.074136    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:47.770286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:47.770246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:48.482052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:48.482018    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.005205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:50.005159    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:51.462865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:51.462274    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:52.816858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:52.816698    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:53.221590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:53.221488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:53.359981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:53.355577    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:54.873024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:54.872990    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.005287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.005233    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.391167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.391133    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"]
Oct 30 21:19:55.392294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392272    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2a74a760-d184-4219-a631-7a07cb625c19" containerName="maas-api"
Oct 30 21:19:55.392362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392299    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a74a760-d184-4219-a631-7a07cb625c19" containerName="maas-api"
Oct 30 21:19:55.392362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392321    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1476163b-023f-43b3-8947-f778ba1aaed0" containerName="payload-pre-processing"
Oct 30 21:19:55.392362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392331    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1476163b-023f-43b3-8947-f778ba1aaed0" containerName="payload-pre-processing"
Oct 30 21:19:55.392465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392362    4549 cpu_manager.go:414] "containerMap: removing container" podUID="82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" containerName="payload-processing"
Oct 30 21:19:55.392465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392370    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" containerName="payload-processing"
Oct 30 21:19:55.392607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392590    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a74a760-d184-4219-a631-7a07cb625c19" containerName="maas-api"
Oct 30 21:19:55.392660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392613    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1476163b-023f-43b3-8947-f778ba1aaed0" containerName="payload-pre-processing"
Oct 30 21:19:55.392660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.392627    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="82cf8eb6-b9d9-4ae1-a72c-2cc0668949a8" containerName="payload-processing"
Oct 30 21:19:55.408409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.408375    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"]
Oct 30 21:19:55.408586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.408483    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.426390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.426357    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c1da2183-openshift-default-dockercfg-gj6jg\""
Oct 30 21:19:55.448582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.448547    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.454083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454047    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.454370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454178    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.454370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454256    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.454572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454552    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.454719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454696    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.454919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454899    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.454979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454967    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.455026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.454993    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.455026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.455018    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsl67\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.570668    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.570755    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.570848    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.570910    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.571023    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.571170    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.571259    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.571306    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.572867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.571337    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsl67\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.591243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.591207    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.602875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.592151    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.602875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.592698    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.602875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.594073    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.602875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.594450    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.602875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.594663    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.636860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.631364    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.637308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.637201    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.643901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.639240    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsl67\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.655853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.647319    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token\") pod \"e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") " pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:55.731194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:55.731126    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:56.287269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.287238    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"]
Oct 30 21:19:56.288149 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:19:56.288121    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44e1c2a_6eef_44fc_817e_83b88588f91b.slice/crio-ea52d112dd29f6108b26cc6eee24629f6a598faaf43e0bb894be07bcc9308637 WatchSource:0}: Error finding container ea52d112dd29f6108b26cc6eee24629f6a598faaf43e0bb894be07bcc9308637: Status 404 returned error can't find the container with id ea52d112dd29f6108b26cc6eee24629f6a598faaf43e0bb894be07bcc9308637
Oct 30 21:19:56.291256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.291221    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:56.291347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.291288    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:56.291347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.291316    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:19:56.575384 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:19:56.575348    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:19:56.723407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.723377    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:56.879011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.878912    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" event={"ID":"b44e1c2a-6eef-44fc-817e-83b88588f91b","Type":"ContainerStarted","Data":"8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"}
Oct 30 21:19:56.879305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.879279    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" event={"ID":"b44e1c2a-6eef-44fc-817e-83b88588f91b","Type":"ContainerStarted","Data":"ea52d112dd29f6108b26cc6eee24629f6a598faaf43e0bb894be07bcc9308637"}
Oct 30 21:19:56.938088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:56.938032    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podStartSLOduration=1.938018557 podStartE2EDuration="1.938018557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:56.933443215 +0000 UTC m=+1179.707843155" watchObservedRunningTime="2026-10-30 21:19:56.938018557 +0000 UTC m=+1179.712418496"
Oct 30 21:19:57.733603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:57.733566    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:57.739460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:57.739437    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:57.886822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:57.886788    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:57.887869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:57.887815    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:19:58.322954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:58.322922    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:59.327053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:59.327024    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:59.463148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:19:59.463113    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:00.013175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:00.013140    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.043196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.034452    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.160860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.160026    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"]
Oct 30 21:20:01.181317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.181283    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"]
Oct 30 21:20:01.181668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.181646    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.196506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.196473    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c1da2183-dockercfg-vjbmj\""
Oct 30 21:20:01.208417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.208385    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c1da2183\""
Oct 30 21:20:01.215657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.215613    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"]
Oct 30 21:20:01.216140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.216113    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.237142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.237110    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.238358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.238207    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"]
Oct 30 21:20:01.323076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.323032    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8nw6\" (UniqueName: \"kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6\") pod \"payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") " pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.453862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.442358    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume\") pod \"payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") " pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.520879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.512238    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.520879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.512523    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dp9z\" (UniqueName: \"kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z\") pod \"payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.621972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.616047    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.683855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.662822    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dp9z\" (UniqueName: \"kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z\") pod \"payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.683855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.663044    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8nw6\" (UniqueName: \"kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6\") pod \"payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") " pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.683855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.663464    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume\") pod \"payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") " pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.683855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.623064    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.710378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.710344    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dp9z\" (UniqueName: \"kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z\") pod \"payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.710822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.710788    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume\") pod \"payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") " pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.728824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.728792    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8nw6\" (UniqueName: \"kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6\") pod \"payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") " pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:01.803579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.803541    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:01.893904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:01.893158    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:02.286930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:02.284914    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:02.701062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:02.701032    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"]
Oct 30 21:20:02.701470 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:20:02.701447    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf662f5ff_5084_4c6a_8749_c1a79feadf1a.slice/crio-386aa885f6828a4715bde8d713eab99c94c3f218da39a72e4c34b30bff37c844 WatchSource:0}: Error finding container 386aa885f6828a4715bde8d713eab99c94c3f218da39a72e4c34b30bff37c844: Status 404 returned error can't find the container with id 386aa885f6828a4715bde8d713eab99c94c3f218da39a72e4c34b30bff37c844
Oct 30 21:20:02.731398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:02.731368    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"]
Oct 30 21:20:02.732255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:20:02.732229    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d1cd9e_5204_4b0d_a225_c897fc0f98ac.slice/crio-a1f63e6087ce9e798937fdc2757ee07e3b679f8f00f701a5ac28e3265dc79772 WatchSource:0}: Error finding container a1f63e6087ce9e798937fdc2757ee07e3b679f8f00f701a5ac28e3265dc79772: Status 404 returned error can't find the container with id a1f63e6087ce9e798937fdc2757ee07e3b679f8f00f701a5ac28e3265dc79772
Oct 30 21:20:03.167326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.167292    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" event={"ID":"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac","Type":"ContainerStarted","Data":"816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b"}
Oct 30 21:20:03.167518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.167344    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" event={"ID":"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac","Type":"ContainerStarted","Data":"a1f63e6087ce9e798937fdc2757ee07e3b679f8f00f701a5ac28e3265dc79772"}
Oct 30 21:20:03.167518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.167367    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:03.168859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.168817    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" event={"ID":"f662f5ff-5084-4c6a-8749-c1a79feadf1a","Type":"ContainerStarted","Data":"7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff"}
Oct 30 21:20:03.168986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.168892    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:03.168986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.168916    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" event={"ID":"f662f5ff-5084-4c6a-8749-c1a79feadf1a","Type":"ContainerStarted","Data":"386aa885f6828a4715bde8d713eab99c94c3f218da39a72e4c34b30bff37c844"}
Oct 30 21:20:03.221032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.220981    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" podStartSLOduration=2.220966133 podStartE2EDuration="2.220966133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:03.216026397 +0000 UTC m=+1185.990426335" watchObservedRunningTime="2026-10-30 21:20:03.220966133 +0000 UTC m=+1185.995366072"
Oct 30 21:20:03.255973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.255920    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" podStartSLOduration=2.25590456 podStartE2EDuration="2.25590456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:03.254012555 +0000 UTC m=+1186.028412496" watchObservedRunningTime="2026-10-30 21:20:03.25590456 +0000 UTC m=+1186.030304499"
Oct 30 21:20:03.961682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:03.961652    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:05.614620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:05.614581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:06.040226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:06.040130    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:07.207589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:07.207552    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:08.599819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.599780    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:08.724149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.724107    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:08.972066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.971980    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"]
Oct 30 21:20:08.977792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.977762    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.995741    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.995866    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhlrp\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.995918    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.995944    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.996073    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.996104    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.996144    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.996229    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:08.996532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:08.996278    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.002636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.002606    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-ad73a4-openshift-default-dockercfg-qlbcn\""
Oct 30 21:20:09.015389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.015358    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"]
Oct 30 21:20:09.065295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.065239    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165337    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165497    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165602    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhlrp\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165655    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165679    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165800    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165851    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165895    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.173855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.165991    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.210034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.166815    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.216951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.186426    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.217177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.198046    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.236813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.198278    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.237052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.213371    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.237165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.218629    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:09.311585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.311505    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.316257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.316189    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.350854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.347353    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.357951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.357890    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhlrp\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp\") pod \"e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") " pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:09.642087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:09.642049    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:10.034154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.034125    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"]
Oct 30 21:20:10.034930 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:20:10.034901    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3671de_b297_412e_a72b_3d54d7553e3d.slice/crio-a6b8a6422c135b8991dad5bf1e7c6f1583b92669d272035dec7168711f591aaf WatchSource:0}: Error finding container a6b8a6422c135b8991dad5bf1e7c6f1583b92669d272035dec7168711f591aaf: Status 404 returned error can't find the container with id a6b8a6422c135b8991dad5bf1e7c6f1583b92669d272035dec7168711f591aaf
Oct 30 21:20:10.038075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.038033    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:10.038161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.038143    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:10.038204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.038193    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:10.325062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.325022    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" event={"ID":"ad3671de-b297-412e-a72b-3d54d7553e3d","Type":"ContainerStarted","Data":"a82af0e983e1ea6bb8646f1aedf8e6efae2a0ccb0e0326e6821581fee2fbc0ee"}
Oct 30 21:20:10.325269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.325091    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" event={"ID":"ad3671de-b297-412e-a72b-3d54d7553e3d","Type":"ContainerStarted","Data":"a6b8a6422c135b8991dad5bf1e7c6f1583b92669d272035dec7168711f591aaf"}
Oct 30 21:20:10.366928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.366879    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podStartSLOduration=2.366864371 podStartE2EDuration="2.366864371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:10.365787419 +0000 UTC m=+1193.140187359" watchObservedRunningTime="2026-10-30 21:20:10.366864371 +0000 UTC m=+1193.141264310"
Oct 30 21:20:10.523145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.523111    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:10.661806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:10.661650    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:11.669131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:11.669104    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:12.159667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:12.159620    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.350383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:12.350354    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:12.351175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:12.351150    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:20:12.834979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:12.834938    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.998372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:12.998329    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:13.929461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:13.929427    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:14.208408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.208316    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:20:14.208408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.208398    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:20:14.366932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.366901    4549 generic.go:356] "Generic (PLEG): container finished" podID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerID="e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510" exitCode=0
Oct 30 21:20:14.367212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.367038    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" event={"ID":"71f46951-9378-4dee-9f2e-42d72da5ee1f","Type":"ContainerDied","Data":"e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"}
Oct 30 21:20:14.367797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.367777    4549 scope.go:122] "RemoveContainer" containerID="e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"
Oct 30 21:20:14.369627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.369593    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:14.369733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.369686    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:14.369800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.369737    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:14.417514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.417469    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:14.845856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:14.841933    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:15.376513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:15.376479    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" event={"ID":"71f46951-9378-4dee-9f2e-42d72da5ee1f","Type":"ContainerStarted","Data":"f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a"}
Oct 30 21:20:15.898587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:15.898506    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:20:15.903210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:15.903175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:15.910325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:15.910305    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:20:15.911214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:15.911189    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:20:15.911320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:15.911238    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:20:16.032851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:16.032793    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:16.336040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:16.336010    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:16.921727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:16.921689    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:20:16.922403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:16.922370    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:20:17.658967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:17.658935    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:20:17.659179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:17.659037    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:20:17.910815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:17.910717    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:20:17.910815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:17.910769    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:20:17.931388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:17.931355    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:18.910786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:18.910749    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:20:18.911017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:18.910821    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:20:19.495287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:19.495256    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:19.910683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:19.910638    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:20:19.910904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:19.910705    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:20:20.910652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:20.910607    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:20:20.911180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:20.910665    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:20:21.157702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:21.157654    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:21.914466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:21.914439    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:20:21.915670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:21.915646    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:20:22.655369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:22.655336    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:22.826852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:22.824784    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:23.082925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:23.073592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.206298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.206260    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-sv8gd-qcn6h"]
Oct 30 21:20:24.211060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.211037    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.227321 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.227287    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-sv8gd-qcn6h"]
Oct 30 21:20:24.234807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.234761    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bkjxf\""
Oct 30 21:20:24.234960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.234809    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:20:24.261981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.261948    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:20:24.297805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.297763    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.297805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.297808    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.298085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.298062    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6nlk\" (UniqueName: \"kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.298277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.298261    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.399929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.399888    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.400134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.399992    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.400134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.400016    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.400257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.400195    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6nlk\" (UniqueName: \"kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.400327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.400303    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.400635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.400619    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.400760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.400741    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.420863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.420815    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6nlk\" (UniqueName: \"kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk\") pod \"periodic-gathering-sv8gd-qcn6h\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") " pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.492683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.492549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.523405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.523364    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:20:24.906356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:24.906326    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-sv8gd-qcn6h"]
Oct 30 21:20:24.907236 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:20:24.907203    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ece9b8_990c_4795_846e_a0edbbd2fbf2.slice/crio-4eaebf05e2f25e912a8a576fa5963d4aa026f0a66add063b7fa00eaef9653161 WatchSource:0}: Error finding container 4eaebf05e2f25e912a8a576fa5963d4aa026f0a66add063b7fa00eaef9653161: Status 404 returned error can't find the container with id 4eaebf05e2f25e912a8a576fa5963d4aa026f0a66add063b7fa00eaef9653161
Oct 30 21:20:25.537496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:25.537456    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h" event={"ID":"76ece9b8-990c-4795-846e-a0edbbd2fbf2","Type":"ContainerStarted","Data":"83f134c9ac021e8e7077195a3c2402dc9526a18848511903b0d8319b99ddb405"}
Oct 30 21:20:25.537985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:25.537521    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h" event={"ID":"76ece9b8-990c-4795-846e-a0edbbd2fbf2","Type":"ContainerStarted","Data":"4eaebf05e2f25e912a8a576fa5963d4aa026f0a66add063b7fa00eaef9653161"}
Oct 30 21:20:25.584042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:25.584002    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h" podStartSLOduration=1.583988892 podStartE2EDuration="1.583988892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:25.580895887 +0000 UTC m=+1208.355295823" watchObservedRunningTime="2026-10-30 21:20:25.583988892 +0000 UTC m=+1208.358388883"
Oct 30 21:20:26.032795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:26.032721    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:26.575272 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:20:26.575236    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:20:27.611017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:27.610981    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:28.912110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:28.912021    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:29.053263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:29.053224    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:29.196147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:29.195989    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:29.543514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:29.543402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:29.670900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:29.670855    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:30.602382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.602347    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"]
Oct 30 21:20:30.606499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.606477    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.629715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.629687    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.629917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.629765    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.629917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.629795    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpt5t\" (UniqueName: \"kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.629917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.629883    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.633091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.633064    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"]
Oct 30 21:20:30.637121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.637095    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-ad73a4-serving-cert\""
Oct 30 21:20:30.725112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.725081    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:30.731877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.731850    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.732033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.731931    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.732033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.731958    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpt5t\" (UniqueName: \"kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.732033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.732031    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.732191 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:20:30.732074    4549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-serving-cert: secret "maas-api-e2e-shared-a-w3-ad73a4-serving-cert" not found
Oct 30 21:20:30.732191 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:20:30.732141    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls podName:c0b3d71c-3e38-46ed-9a74-71e70e7de322 nodeName:}" failed. No retries permitted until 2026-10-30 21:20:31.2321265 +0000 UTC m=+1214.006526430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls") pod "maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" (UID: "c0b3d71c-3e38-46ed-9a74-71e70e7de322") : secret "maas-api-e2e-shared-a-w3-ad73a4-serving-cert" not found
Oct 30 21:20:30.732528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.732507    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.734878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.734813    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:30.760776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:30.760746    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpt5t\" (UniqueName: \"kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:31.056547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:31.051712    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:31.239966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:31.239931    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:31.242575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:31.242543    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:31.523614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:31.523583    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:32.008936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:32.004355    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"]
Oct 30 21:20:32.695733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:32.687462    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:32.708854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:32.705049    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" event={"ID":"c0b3d71c-3e38-46ed-9a74-71e70e7de322","Type":"ContainerStarted","Data":"5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406"}
Oct 30 21:20:32.708854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:32.705128    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" event={"ID":"c0b3d71c-3e38-46ed-9a74-71e70e7de322","Type":"ContainerStarted","Data":"12bdeebdc4dd10fd79b429336d556b85c18cde3a992f4eabd0ded57e7b99adb8"}
Oct 30 21:20:32.708854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:32.705947    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:32.773007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:32.770081    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" podStartSLOduration=2.332187391 podStartE2EDuration="2.770056693s" totalImagesPullingTime="437.869302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:32.76320585 +0000 UTC m=+1215.537605789" watchObservedRunningTime="2026-10-30 21:20:32.770056693 +0000 UTC m=+1215.544456632"
Oct 30 21:20:34.327554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:34.327495    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:35.921485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:35.921451    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:37.522854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:37.522295    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:38.717895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:38.717867    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:20:39.116193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:39.116151    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:39.963607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:39.963570    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:40.371251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:40.371216    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:40.722186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:40.722101    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:40.858929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:40.858881    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:42.382641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.382597    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:42.581714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.581678    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"]
Oct 30 21:20:42.586473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.586454    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.611722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.611675    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"]
Oct 30 21:20:42.612555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.612529    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c1da2183\"/\"e2e-lifecycle-model-c1da2183-kserve-self-signed-certs\""
Oct 30 21:20:42.612734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.612651    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c1da2183\"/\"default-dockercfg-rc26c\""
Oct 30 21:20:42.615933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.615906    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c1da2183\"/\"kube-root-ca.crt\""
Oct 30 21:20:42.616054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.616017    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c1da2183\"/\"openshift-service-ca.crt\""
Oct 30 21:20:42.698688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.698605    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:42.747111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.747082    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.747307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.747201    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.747307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.747245    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.747421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.747315    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6j6\" (UniqueName: \"kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.747475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.747446    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.747539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.747497    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.850028    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.850178    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.850471    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.850582    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.850628    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.850696    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6j6\" (UniqueName: \"kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.851554    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.853854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.851793    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.884856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.873639    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.885136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.876730    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.910888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.894965    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6j6\" (UniqueName: \"kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.910888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.895544    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs\") pod \"e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") " pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:42.926854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:42.925190    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:43.422547 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:20:43.422515    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod263b186c_9d33_4e08_81b7_fa11f0731942.slice/crio-33939c95a5aebae2e86b514adba63c4493cc111812a75ab40d39420af2065a56 WatchSource:0}: Error finding container 33939c95a5aebae2e86b514adba63c4493cc111812a75ab40d39420af2065a56: Status 404 returned error can't find the container with id 33939c95a5aebae2e86b514adba63c4493cc111812a75ab40d39420af2065a56
Oct 30 21:20:43.422906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:43.422881    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"]
Oct 30 21:20:43.794765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:43.794674    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" event={"ID":"263b186c-9d33-4e08-81b7-fa11f0731942","Type":"ContainerStarted","Data":"172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f"}
Oct 30 21:20:43.794765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:43.794725    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" event={"ID":"263b186c-9d33-4e08-81b7-fa11f0731942","Type":"ContainerStarted","Data":"33939c95a5aebae2e86b514adba63c4493cc111812a75ab40d39420af2065a56"}
Oct 30 21:20:43.836328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:43.836278    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podStartSLOduration=1.83626328 podStartE2EDuration="1.83626328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:43.832802608 +0000 UTC m=+1226.607202548" watchObservedRunningTime="2026-10-30 21:20:43.83626328 +0000 UTC m=+1226.610663219"
Oct 30 21:20:43.926345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:43.926309    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:44.011437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:44.011401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.986107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:44.986070    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:45.547854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:45.543440    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:47.041728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:47.041695    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:49.257425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:49.257366    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.001909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:51.001630    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.134404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:51.134324    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:52.942910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:52.941729    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:53.136216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:53.136176    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:53.215913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:53.207715    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:53.943990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:53.943960    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:20:55.290414 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:55.290375    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:56.621373 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:20:56.621328    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:20:56.947473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:56.947379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.393030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:58.392977    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:20:58.393528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:58.393065    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:20:58.540705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:58.540666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.681058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:58.676650    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.928027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:58.927987    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:20:58.928226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:58.928080    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:20:59.007444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:59.007353    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:59.475212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:20:59.475181    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:00.366616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:00.366532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:00.541855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:00.535622    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:01.989155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:01.989113    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:02.251093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:02.251056    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:02.557507 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:21:02.557469    4549 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@10-21926-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3671de_b297_412e_a72b_3d54d7553e3d.slice/crio-a82af0e983e1ea6bb8646f1aedf8e6efae2a0ccb0e0326e6821581fee2fbc0ee.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:21:03.165191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.165158    4549 generic.go:356] "Generic (PLEG): container finished" podID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerID="8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149" exitCode=0
Oct 30 21:21:03.165750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.165234    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" event={"ID":"b44e1c2a-6eef-44fc-817e-83b88588f91b","Type":"ContainerDied","Data":"8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"}
Oct 30 21:21:03.165750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.165714    4549 scope.go:122] "RemoveContainer" containerID="8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"
Oct 30 21:21:03.167251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.167228    4549 generic.go:356] "Generic (PLEG): container finished" podID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerID="a82af0e983e1ea6bb8646f1aedf8e6efae2a0ccb0e0326e6821581fee2fbc0ee" exitCode=0
Oct 30 21:21:03.167342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.167301    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" event={"ID":"ad3671de-b297-412e-a72b-3d54d7553e3d","Type":"ContainerDied","Data":"a82af0e983e1ea6bb8646f1aedf8e6efae2a0ccb0e0326e6821581fee2fbc0ee"}
Oct 30 21:21:03.167777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.167760    4549 scope.go:122] "RemoveContainer" containerID="a82af0e983e1ea6bb8646f1aedf8e6efae2a0ccb0e0326e6821581fee2fbc0ee"
Oct 30 21:21:03.167918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.167888    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:21:03.168007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.167989    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:21:03.168052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.168039    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:21:03.169653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.169620    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:21:03.169726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.169710    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:21:03.169773 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.169759    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:21:03.205004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.204973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:03.362852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:03.360057    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:04.104936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.104901    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:04.250880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.241856    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" event={"ID":"b44e1c2a-6eef-44fc-817e-83b88588f91b","Type":"ContainerStarted","Data":"a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071"}
Oct 30 21:21:04.264205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.264112    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" event={"ID":"ad3671de-b297-412e-a72b-3d54d7553e3d","Type":"ContainerStarted","Data":"8478a026758460e6f9440444ef0dc34486838a5941f62ad7a7b03a9d153ad7ae"}
Oct 30 21:21:04.662089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.662054    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:21:04.662377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.662334    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:04.662534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.662426    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:04.734702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.734672    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:21:04.735875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.735302    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:04.735875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.735624    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:04.935397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:04.935268    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.066014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:05.065980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.666012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:05.665975    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:05.666629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:05.666597    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:05.747431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:05.747399    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:05.747761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:05.747694    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:06.161067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:06.161034    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:06.302168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:06.297379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:06.662068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:06.662030    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:06.662230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:06.662114    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:06.739857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:06.735093    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:06.739857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:06.735163    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:07.676411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.676326    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:07.676732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.676697    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:07.742450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.742406    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:07.743125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.743081    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:07.755157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.755114    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:07.784087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.784037    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:07.852007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.851145    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:07.956495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:07.956381    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:08.012019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.011967    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:08.084898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.084860    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:08.243290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.242935    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:08.269000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.268939    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:08.678304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.678261    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:08.678592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.678562    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:08.702752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.702704    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:08.737203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.737168    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:08.737469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.737441    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:08.747356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.747317    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:21:08.759516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.759469    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:08.760166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.760138    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:21:08.791948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.791903    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:21:08.792575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:08.792445    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:21:09.031371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.031265    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:09.045980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.045951    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:21:09.304926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.289237    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:21:09.304926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.290346    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:21:09.443856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.434171    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:09.679853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.679282    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:09.679853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.679359    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:09.792867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.791537    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:21:09.828575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.828183    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:21:09.964746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.835475    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.969887    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.863794    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.971222    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.934099    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.865616    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:09.854982    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:10.085858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.036027    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:10.295909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.291917    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:10.554555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.554456    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:10.630343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.624378    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:10.782795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.782760    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:10.783109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.783079    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:10.809859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.809756    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:10.810400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.810371    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:10.860212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.856226    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:10.963336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.963291    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:10.965317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:10.965285    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:11.033863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.031968    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:11.033863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.032477    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:21:11.047896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.047855    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:21:11.235948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.207163    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:11.659853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.632940    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:21:11.659853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.636340    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:11.711858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.665447    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:11.711858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.665508    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:11.739859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.736681    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:11.739859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.736740    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:11.858023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.857962    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:11.872034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.871981    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:21:11.964066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.963953    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:11.967616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:11.966650    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:21:12.056862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.051401    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:12.074884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.054612    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:12.219329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.219195    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:12.246923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.243909    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:12.354073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.352944    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:21:12.385619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.385588    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:21:12.420854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.413204    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:21:12.563736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.563655    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:21:12.593861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.590092    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:12.736191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.712199    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 21:21:12.736191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.712253    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 21:21:12.737453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.737421    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 21:21:12.737655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.737630    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 21:21:12.749928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.738918    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:12.871396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:12.871348    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:13.064704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.064669    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:21:13.333908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.144114    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:21:13.336080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.336053    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:21:13.337370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.259663    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:21:13.447369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.447246    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:21:13.463278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.463255    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:21:13.466538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.466517    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:21:13.476989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.476960    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:21:13.813527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.813440    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:21:13.814945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:13.814921    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:21:14.335003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:14.334957    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:14.363325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:14.363298    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:21:14.368444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:14.368408    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:21:14.493211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:14.493182    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:14.712576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:14.712479    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:14.878855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:14.875440    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:15.471695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:15.471663    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:15.682857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:15.677941    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:16.127265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:16.127187    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:16.299459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:16.299380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:16.430206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:16.430104    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:17.621589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:17.621551    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:18.009376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:18.009287    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:18.144184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:18.144145    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.480812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:19.480771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.653020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:19.652985    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.940872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:19.938242    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:20.282920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:20.282792    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.281945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.281912    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.346344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.346316    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"]
Oct 30 21:21:21.351065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.351043    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.363925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.363897    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"]
Oct 30 21:21:21.424870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.422144    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-a4782a\""
Oct 30 21:21:21.424870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.423098    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-a4782a-dockercfg-t4clj\""
Oct 30 21:21:21.451855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.449378    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.471670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.471630    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj2q6\" (UniqueName: \"kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6\") pod \"payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.498865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.480000    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.594861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.594553    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj2q6\" (UniqueName: \"kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6\") pod \"payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.594861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.594724    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.628858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.595872    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.654677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.654645    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj2q6\" (UniqueName: \"kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6\") pod \"payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.727424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.724326    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:21.869574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:21.869436    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:22.452857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:22.445637    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"]
Oct 30 21:21:22.465175 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:21:22.465137    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f333559_04dc_4362_82d8_8bbdec4f743c.slice/crio-f2cac8efa61dff0e0202390d23216118216a653e53f24eb12f63f83f3e59357f WatchSource:0}: Error finding container f2cac8efa61dff0e0202390d23216118216a653e53f24eb12f63f83f3e59357f: Status 404 returned error can't find the container with id f2cac8efa61dff0e0202390d23216118216a653e53f24eb12f63f83f3e59357f
Oct 30 21:21:22.624255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:22.624225    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" event={"ID":"8f333559-04dc-4362-82d8-8bbdec4f743c","Type":"ContainerStarted","Data":"c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b"}
Oct 30 21:21:22.624374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:22.624287    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" event={"ID":"8f333559-04dc-4362-82d8-8bbdec4f743c","Type":"ContainerStarted","Data":"f2cac8efa61dff0e0202390d23216118216a653e53f24eb12f63f83f3e59357f"}
Oct 30 21:21:22.624374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:22.624318    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:22.664625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:22.664535    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" podStartSLOduration=1.664520631 podStartE2EDuration="1.664520631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:21:22.660965999 +0000 UTC m=+1265.435365938" watchObservedRunningTime="2026-10-30 21:21:22.664520631 +0000 UTC m=+1265.438920571"
Oct 30 21:21:23.385645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:23.385613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:24.794993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:24.794963    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:26.442862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:26.441427    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:26.577701 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:21:26.577667    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:21:27.800911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:27.800881    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.931996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:27.931933    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:28.067785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:28.066262    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:29.971239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:29.971152    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:31.612214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:31.612175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:33.076214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:33.076179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:33.636266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:33.636239    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:21:34.508191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:34.508160    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:35.969001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:35.968966    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:37.391524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:37.391488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:37.535460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:37.535405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:39.174757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.174721    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"]
Oct 30 21:21:39.179464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.179440    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.205421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.205387    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0e7510-serving-cert\""
Oct 30 21:21:39.211960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.211927    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"]
Oct 30 21:21:39.278760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.278720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:39.290190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.290152    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.290390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.290350    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwphj\" (UniqueName: \"kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.290530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.290508    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.290780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.290757    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.392845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.392788    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwphj\" (UniqueName: \"kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.393060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.392932    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.393130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.393115    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.393249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.393233    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.393763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.393739    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.395664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.395645    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.395902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.395879    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.404473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.404447    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:39.412658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.412634    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwphj\" (UniqueName: \"kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj\") pod \"maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.499721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.499631    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:39.535019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:39.534980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:40.013474 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:21:40.013428    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf524c2ca_c055_443b_9d1e_594e6a761c2e.slice/crio-6bd3d46592521a90e1814d944558c37f14ddad53a5fe05741f5cf6a1611e925e WatchSource:0}: Error finding container 6bd3d46592521a90e1814d944558c37f14ddad53a5fe05741f5cf6a1611e925e: Status 404 returned error can't find the container with id 6bd3d46592521a90e1814d944558c37f14ddad53a5fe05741f5cf6a1611e925e
Oct 30 21:21:40.014231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:40.014195    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"]
Oct 30 21:21:40.040898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:40.020304    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:21:40.885315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:40.885274    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" event={"ID":"f524c2ca-c055-443b-9d1e-594e6a761c2e","Type":"ContainerStarted","Data":"d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2"}
Oct 30 21:21:40.885873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:40.885336    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" event={"ID":"f524c2ca-c055-443b-9d1e-594e6a761c2e","Type":"ContainerStarted","Data":"6bd3d46592521a90e1814d944558c37f14ddad53a5fe05741f5cf6a1611e925e"}
Oct 30 21:21:40.885873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:40.885498    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:40.944992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:40.944646    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" podStartSLOduration=1.442892335 podStartE2EDuration="1.944624795s" totalImagesPullingTime="501.73246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:21:40.939348859 +0000 UTC m=+1283.713748799" watchObservedRunningTime="2026-10-30 21:21:40.944624795 +0000 UTC m=+1283.719024739"
Oct 30 21:21:41.042269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:41.042230    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:42.448347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:42.448314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:43.762477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.762435    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"]
Oct 30 21:21:43.767337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.767314    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.781791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.781759    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-a4782a-serving-cert\""
Oct 30 21:21:43.801149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.801113    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"]
Oct 30 21:21:43.851516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.851468    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.851727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.851701    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.851798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.851772    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.851995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.851973    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcwmg\" (UniqueName: \"kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.861367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.861330    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:43.954189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.954147    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.954410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.954294    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.954410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.954353    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.954547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.954431    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcwmg\" (UniqueName: \"kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.955015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.954990    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.957172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.957144    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.957292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.957270    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:43.986853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:43.975695    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcwmg\" (UniqueName: \"kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg\") pod \"maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:44.007873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:44.007786    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.099618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:44.098271    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:44.524767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:44.524738    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"]
Oct 30 21:21:44.527753 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:21:44.527725    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a572e56_2552_49be_94ee_00233fdae91a.slice/crio-2c08822a1034ce600712f7ec69fc791188da9d6a5aa425e3631d9d6c4d5cd7e9 WatchSource:0}: Error finding container 2c08822a1034ce600712f7ec69fc791188da9d6a5aa425e3631d9d6c4d5cd7e9: Status 404 returned error can't find the container with id 2c08822a1034ce600712f7ec69fc791188da9d6a5aa425e3631d9d6c4d5cd7e9
Oct 30 21:21:44.920010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:44.919973    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4" event={"ID":"1a572e56-2552-49be-94ee-00233fdae91a","Type":"ContainerStarted","Data":"2c08822a1034ce600712f7ec69fc791188da9d6a5aa425e3631d9d6c4d5cd7e9"}
Oct 30 21:21:45.901339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:45.901058    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:45.946214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:45.946178    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4" event={"ID":"1a572e56-2552-49be-94ee-00233fdae91a","Type":"ContainerStarted","Data":"bfceac72fa027a8dbaa150f6f468fafdfd21c58ba46c5592620c04c4f5777eba"}
Oct 30 21:21:45.946971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:45.946952    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:45.990910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:45.990793    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4" podStartSLOduration=2.6601273770000002 podStartE2EDuration="2.990778041s" totalImagesPullingTime="330.650664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:21:45.988550808 +0000 UTC m=+1288.762950747" watchObservedRunningTime="2026-10-30 21:21:45.990778041 +0000 UTC m=+1288.765178024"
Oct 30 21:21:46.899509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:46.899480    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:21:47.433771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:47.433737    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:48.863144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:48.863103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:49.841788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:49.841747    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:50.385161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:50.385130    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:51.835947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:51.835907    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:51.992089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:51.992057    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:21:53.310524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:53.310476    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:55.145406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:55.145367    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:56.577143 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:21:56.577098    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:21:56.746218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:56.746170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:58.341101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:58.341066    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.089552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:59.089515    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.880875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:59.880812    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.976119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:21:59.976088    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-e7b664"]
Oct 30 21:22:00.012288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.011335    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-e7b664"]
Oct 30 21:22:00.012288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.011464    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:00.066648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.066618    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxp2m\""
Oct 30 21:22:00.107481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.107436    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlfng\" (UniqueName: \"kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng\") pod \"test-curl-7401-e7b664\" (UID: \"b22bde60-735d-4959-8534-b0dda0e60a63\") " pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:00.210542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.210440    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlfng\" (UniqueName: \"kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng\") pod \"test-curl-7401-e7b664\" (UID: \"b22bde60-735d-4959-8534-b0dda0e60a63\") " pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:00.230736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.230697    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlfng\" (UniqueName: \"kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng\") pod \"test-curl-7401-e7b664\" (UID: \"b22bde60-735d-4959-8534-b0dda0e60a63\") " pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:00.352373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.352339    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:00.660718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:00.660674    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-e7b664"]
Oct 30 21:22:00.660948 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:00.660922    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb22bde60_735d_4959_8534_b0dda0e60a63.slice/crio-f4eb890deb9743335450577cba237da815187f4677a5ecaea60ad55e00664767 WatchSource:0}: Error finding container f4eb890deb9743335450577cba237da815187f4677a5ecaea60ad55e00664767: Status 404 returned error can't find the container with id f4eb890deb9743335450577cba237da815187f4677a5ecaea60ad55e00664767
Oct 30 21:22:01.130404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:01.130367    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-e7b664" event={"ID":"b22bde60-735d-4959-8534-b0dda0e60a63","Type":"ContainerStarted","Data":"cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4"}
Oct 30 21:22:01.130933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:01.130415    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-e7b664" event={"ID":"b22bde60-735d-4959-8534-b0dda0e60a63","Type":"ContainerStarted","Data":"f4eb890deb9743335450577cba237da815187f4677a5ecaea60ad55e00664767"}
Oct 30 21:22:01.170917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:01.170866    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7401-e7b664" podStartSLOduration=2.170851192 podStartE2EDuration="2.170851192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:01.166157093 +0000 UTC m=+1303.940557032" watchObservedRunningTime="2026-10-30 21:22:01.170851192 +0000 UTC m=+1303.945251128"
Oct 30 21:22:01.529758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:01.529675    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.202046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.187366    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-e7b664"]
Oct 30 21:22:02.202046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.195895    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-e7b664"]
Oct 30 21:22:02.211936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.202989    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-50dacb"]
Oct 30 21:22:02.211936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.204263    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" containerName="test-curl-7401-e7b664"
Oct 30 21:22:02.211936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.204284    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" containerName="test-curl-7401-e7b664"
Oct 30 21:22:02.211936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.204484    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" containerName="test-curl-7401-e7b664"
Oct 30 21:22:02.235969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.212297    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:02.350994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.350383    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8lh4\" (UniqueName: \"kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4\") pod \"test-curl-7401-50dacb\" (UID: \"e89bfd90-b934-4352-a705-a3b64d419e1a\") " pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:02.350994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.350888    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-50dacb"]
Oct 30 21:22:02.451989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.451957    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8lh4\" (UniqueName: \"kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4\") pod \"test-curl-7401-50dacb\" (UID: \"e89bfd90-b934-4352-a705-a3b64d419e1a\") " pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:02.473779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.473704    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8lh4\" (UniqueName: \"kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4\") pod \"test-curl-7401-50dacb\" (UID: \"e89bfd90-b934-4352-a705-a3b64d419e1a\") " pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:02.705499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:02.705455    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:03.257163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:03.243151    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-50dacb"]
Oct 30 21:22:03.372790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:03.372731    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7401-e7b664" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" containerName="test-curl-7401-e7b664" containerID="cri-o://cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4" gracePeriod=2
Oct 30 21:22:03.434860 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:03.428159    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-e7b664\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:03.455223 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:03.455174    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode89bfd90_b934_4352_a705_a3b64d419e1a.slice/crio-d4ca73f4cea0a853166d7c63190662e0d03c962273140d98e47b38bab1cc0312 WatchSource:0}: Error finding container d4ca73f4cea0a853166d7c63190662e0d03c962273140d98e47b38bab1cc0312: Status 404 returned error can't find the container with id d4ca73f4cea0a853166d7c63190662e0d03c962273140d98e47b38bab1cc0312
Oct 30 21:22:04.345388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:04.345348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.440855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:04.439672    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-50dacb" event={"ID":"e89bfd90-b934-4352-a705-a3b64d419e1a","Type":"ContainerStarted","Data":"ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61"}
Oct 30 21:22:04.440855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:04.439740    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-50dacb" event={"ID":"e89bfd90-b934-4352-a705-a3b64d419e1a","Type":"ContainerStarted","Data":"d4ca73f4cea0a853166d7c63190662e0d03c962273140d98e47b38bab1cc0312"}
Oct 30 21:22:04.456856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:04.450187    4549 generic.go:356] "Generic (PLEG): container finished" podID="76ece9b8-990c-4795-846e-a0edbbd2fbf2" containerID="83f134c9ac021e8e7077195a3c2402dc9526a18848511903b0d8319b99ddb405" exitCode=0
Oct 30 21:22:04.456856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:04.450306    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h" event={"ID":"76ece9b8-990c-4795-846e-a0edbbd2fbf2","Type":"ContainerDied","Data":"83f134c9ac021e8e7077195a3c2402dc9526a18848511903b0d8319b99ddb405"}
Oct 30 21:22:04.546188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:04.523602    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7401-50dacb" podStartSLOduration=2.523581328 podStartE2EDuration="2.523581328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:04.498423419 +0000 UTC m=+1307.272823359" watchObservedRunningTime="2026-10-30 21:22:04.523581328 +0000 UTC m=+1307.297981269"
Oct 30 21:22:05.376524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.376493    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-50dacb"]
Oct 30 21:22:05.383029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.383001    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-50dacb"]
Oct 30 21:22:05.537985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.537950    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-658e5d"]
Oct 30 21:22:05.539230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.539202    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" containerName="test-curl-7401-50dacb"
Oct 30 21:22:05.539230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.539229    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" containerName="test-curl-7401-50dacb"
Oct 30 21:22:05.539475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.539461    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" containerName="test-curl-7401-50dacb"
Oct 30 21:22:05.553375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.553340    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-658e5d"]
Oct 30 21:22:05.553570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.553535    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:05.629961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.629815    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jffjk\" (UniqueName: \"kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk\") pod \"test-curl-7401-658e5d\" (UID: \"594ac4bb-f2fe-409f-be6b-b1392cb817aa\") " pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:05.669790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.669760    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:22:05.765888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.758347    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jffjk\" (UniqueName: \"kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk\") pod \"test-curl-7401-658e5d\" (UID: \"594ac4bb-f2fe-409f-be6b-b1392cb817aa\") " pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:05.929913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.904404    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path\") pod \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") "
Oct 30 21:22:05.944866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.934179    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle\") pod \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") "
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.933935    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path" pod "76ece9b8-990c-4795-846e-a0edbbd2fbf2" (UID: "76ece9b8-990c-4795-846e-a0edbbd2fbf2"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.961349    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle" pod "76ece9b8-990c-4795-846e-a0edbbd2fbf2" (UID: "76ece9b8-990c-4795-846e-a0edbbd2fbf2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.985372    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle\") pod \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") "
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.985431    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk\" (UniqueName: \"kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk\") pod \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\" (UID: \"76ece9b8-990c-4795-846e-a0edbbd2fbf2\") "
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.986351    4549 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/76ece9b8-990c-4795-846e-a0edbbd2fbf2-archives-path\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.986373    4549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-trusted-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:06.005858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:05.987360    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle" pod "76ece9b8-990c-4795-846e-a0edbbd2fbf2" (UID: "76ece9b8-990c-4795-846e-a0edbbd2fbf2"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:06.023490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.023390    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk" pod "76ece9b8-990c-4795-846e-a0edbbd2fbf2" (UID: "76ece9b8-990c-4795-846e-a0edbbd2fbf2"). InnerVolumeSpecName "kube-api-access-m6nlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:06.042563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.039610    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jffjk\" (UniqueName: \"kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk\") pod \"test-curl-7401-658e5d\" (UID: \"594ac4bb-f2fe-409f-be6b-b1392cb817aa\") " pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:06.120859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.113232    4549 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76ece9b8-990c-4795-846e-a0edbbd2fbf2-service-ca-bundle\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:06.127309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.127279    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6nlk\" (UniqueName: \"kubernetes.io/projected/76ece9b8-990c-4795-846e-a0edbbd2fbf2-kube-api-access-m6nlk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:06.199911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.199878    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:06.217747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.217693    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:06.261663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.261628    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng\" (UniqueName: \"kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng\") pod \"b22bde60-735d-4959-8534-b0dda0e60a63\" (UID: \"b22bde60-735d-4959-8534-b0dda0e60a63\") "
Oct 30 21:22:06.263881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.263854    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng" pod "b22bde60-735d-4959-8534-b0dda0e60a63" (UID: "b22bde60-735d-4959-8534-b0dda0e60a63"). InnerVolumeSpecName "kube-api-access-hlfng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:06.267639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.267615    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:06.388222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.388174    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlfng\" (UniqueName: \"kubernetes.io/projected/b22bde60-735d-4959-8534-b0dda0e60a63-kube-api-access-hlfng\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:06.493540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.471065    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:06.500428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.500394    4549 generic.go:356] "Generic (PLEG): container finished" podID="b22bde60-735d-4959-8534-b0dda0e60a63" containerID="cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4" exitCode=137
Oct 30 21:22:06.500732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.500710    4549 scope.go:122] "RemoveContainer" containerID="cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4"
Oct 30 21:22:06.501099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.501076    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:06.506317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.506272    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7401-50dacb" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" containerName="test-curl-7401-50dacb" containerID="cri-o://ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61" gracePeriod=2
Oct 30 21:22:06.508331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.506514    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h"
Oct 30 21:22:06.515843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.506616    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-sv8gd-qcn6h" event={"ID":"76ece9b8-990c-4795-846e-a0edbbd2fbf2","Type":"ContainerDied","Data":"4eaebf05e2f25e912a8a576fa5963d4aa026f0a66add063b7fa00eaef9653161"}
Oct 30 21:22:06.515843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.513575    4549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4eaebf05e2f25e912a8a576fa5963d4aa026f0a66add063b7fa00eaef9653161"
Oct 30 21:22:06.521467 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:06.521182    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-50dacb\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:06.526141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.526105    4549 scope.go:122] "RemoveContainer" containerID="cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4"
Oct 30 21:22:06.526787 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:06.526703    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4\": container with ID starting with cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4 not found: ID does not exist" containerID="cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4"
Oct 30 21:22:06.526787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.526750    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4"} err="failed to get container status \"cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4\": rpc error: code = NotFound desc = could not find container \"cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4\": container with ID starting with cf4de621ff4b77332062ebe4d586cce145dd01dc7da604480dc4f228b1f62ca4 not found: ID does not exist"
Oct 30 21:22:06.534280 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:06.534251    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-e7b664\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:06.547457 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:06.547418    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-50dacb\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:06.560521 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:06.560480    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-e7b664\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:06.573582 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:06.573524    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-50dacb\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:06.692533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:06.692500    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-658e5d"]
Oct 30 21:22:06.693505 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:06.693448    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod594ac4bb_f2fe_409f_be6b_b1392cb817aa.slice/crio-a3589d3d4edeae8ea5ff108cab95a08349a7e9227d6bd0437b86409c2da06f3d WatchSource:0}: Error finding container a3589d3d4edeae8ea5ff108cab95a08349a7e9227d6bd0437b86409c2da06f3d: Status 404 returned error can't find the container with id a3589d3d4edeae8ea5ff108cab95a08349a7e9227d6bd0437b86409c2da06f3d
Oct 30 21:22:07.360775 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:07.360726    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-e7b664\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" pod="opendatahub/test-curl-7401-e7b664"
Oct 30 21:22:07.386661 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:07.374306    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-50dacb\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:07.386661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:07.378409    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b22bde60-735d-4959-8534-b0dda0e60a63" path="/var/lib/kubelet/pods/b22bde60-735d-4959-8534-b0dda0e60a63/volumes"
Oct 30 21:22:07.465760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:07.465706    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:07.519182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:07.519146    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-658e5d" event={"ID":"594ac4bb-f2fe-409f-be6b-b1392cb817aa","Type":"ContainerStarted","Data":"78a592d97851d272c76d9640f7c8cde341a1f56e5fcafbf7fc14da7824e84911"}
Oct 30 21:22:07.519368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:07.519195    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-658e5d" event={"ID":"594ac4bb-f2fe-409f-be6b-b1392cb817aa","Type":"ContainerStarted","Data":"a3589d3d4edeae8ea5ff108cab95a08349a7e9227d6bd0437b86409c2da06f3d"}
Oct 30 21:22:07.555593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:07.555530    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7401-658e5d" podStartSLOduration=2.5555138189999997 podStartE2EDuration="2.555513819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:07.554614605 +0000 UTC m=+1310.329014548" watchObservedRunningTime="2026-10-30 21:22:07.555513819 +0000 UTC m=+1310.329913771"
Oct 30 21:22:07.593321 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:07.593285    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:08.099862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.096671    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:08.756923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.756895    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:08.817470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.817438    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-658e5d"]
Oct 30 21:22:08.827418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.827389    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-658e5d"]
Oct 30 21:22:08.852395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.852359    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4\" (UniqueName: \"kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4\") pod \"e89bfd90-b934-4352-a705-a3b64d419e1a\" (UID: \"e89bfd90-b934-4352-a705-a3b64d419e1a\") "
Oct 30 21:22:08.856062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.856028    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4" pod "e89bfd90-b934-4352-a705-a3b64d419e1a" (UID: "e89bfd90-b934-4352-a705-a3b64d419e1a"). InnerVolumeSpecName "kube-api-access-f8lh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:08.954502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.954467    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8lh4\" (UniqueName: \"kubernetes.io/projected/e89bfd90-b934-4352-a705-a3b64d419e1a-kube-api-access-f8lh4\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:08.981207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.981174    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-d7235c"]
Oct 30 21:22:08.982044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.982020    4549 cpu_manager.go:414] "containerMap: removing container" podUID="76ece9b8-990c-4795-846e-a0edbbd2fbf2" containerName="insights-gathering"
Oct 30 21:22:08.982044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.982039    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76ece9b8-990c-4795-846e-a0edbbd2fbf2" containerName="insights-gathering"
Oct 30 21:22:08.982222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.982058    4549 cpu_manager.go:414] "containerMap: removing container" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" containerName="test-curl-7401-658e5d"
Oct 30 21:22:08.982222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.982063    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" containerName="test-curl-7401-658e5d"
Oct 30 21:22:08.982222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.982210    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="76ece9b8-990c-4795-846e-a0edbbd2fbf2" containerName="insights-gathering"
Oct 30 21:22:08.982222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.982218    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" containerName="test-curl-7401-658e5d"
Oct 30 21:22:08.985777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:08.985758    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:09.000345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.000321    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-d7235c"]
Oct 30 21:22:09.055514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.055481    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkddt\" (UniqueName: \"kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt\") pod \"test-curl-7401-d7235c\" (UID: \"872c3995-3fa2-4061-a273-607174ccefc8\") " pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:09.157285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.157189    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkddt\" (UniqueName: \"kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt\") pod \"test-curl-7401-d7235c\" (UID: \"872c3995-3fa2-4061-a273-607174ccefc8\") " pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:09.177698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.177669    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkddt\" (UniqueName: \"kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt\") pod \"test-curl-7401-d7235c\" (UID: \"872c3995-3fa2-4061-a273-607174ccefc8\") " pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:09.300221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.300191    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:09.348465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.348419    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e89bfd90-b934-4352-a705-a3b64d419e1a" path="/var/lib/kubelet/pods/e89bfd90-b934-4352-a705-a3b64d419e1a/volumes"
Oct 30 21:22:09.561882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.561799    4549 generic.go:356] "Generic (PLEG): container finished" podID="e89bfd90-b934-4352-a705-a3b64d419e1a" containerID="ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61" exitCode=137
Oct 30 21:22:09.561882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.561868    4549 scope.go:122] "RemoveContainer" containerID="ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61"
Oct 30 21:22:09.561882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.561885    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-50dacb"
Oct 30 21:22:09.562167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.562148    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7401-658e5d" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" containerName="test-curl-7401-658e5d" containerID="cri-o://78a592d97851d272c76d9640f7c8cde341a1f56e5fcafbf7fc14da7824e84911" gracePeriod=2
Oct 30 21:22:09.575434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.575412    4549 scope.go:122] "RemoveContainer" containerID="ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61"
Oct 30 21:22:09.575953 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:09.575914    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61\": container with ID starting with ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61 not found: ID does not exist" containerID="ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61"
Oct 30 21:22:09.576057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.575965    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61"} err="failed to get container status \"ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61\": rpc error: code = NotFound desc = could not find container \"ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61\": container with ID starting with ad7728d3f8cde1b3fa36d0eb1e036b372d8dc035118e604f106b674168732e61 not found: ID does not exist"
Oct 30 21:22:09.608331 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:09.608297    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod872c3995_3fa2_4061_a273_607174ccefc8.slice/crio-f1ad7d2095d4de5f723f41a9a28df6de9fa3e5840dd6557872bea71c21b10353 WatchSource:0}: Error finding container f1ad7d2095d4de5f723f41a9a28df6de9fa3e5840dd6557872bea71c21b10353: Status 404 returned error can't find the container with id f1ad7d2095d4de5f723f41a9a28df6de9fa3e5840dd6557872bea71c21b10353
Oct 30 21:22:09.609064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.609040    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-d7235c"]
Oct 30 21:22:09.929333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:09.929302    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.372228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:10.372193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.576055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:10.575262    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-d7235c" event={"ID":"872c3995-3fa2-4061-a273-607174ccefc8","Type":"ContainerStarted","Data":"5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c"}
Oct 30 21:22:10.576055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:10.575405    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-d7235c" event={"ID":"872c3995-3fa2-4061-a273-607174ccefc8","Type":"ContainerStarted","Data":"f1ad7d2095d4de5f723f41a9a28df6de9fa3e5840dd6557872bea71c21b10353"}
Oct 30 21:22:10.635754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:10.635632    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7401-d7235c" podStartSLOduration=2.635614423 podStartE2EDuration="2.635614423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:10.635299673 +0000 UTC m=+1313.409699613" watchObservedRunningTime="2026-10-30 21:22:10.635614423 +0000 UTC m=+1313.410014363"
Oct 30 21:22:11.168984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.168934    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:11.556195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.556117    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"]
Oct 30 21:22:11.565011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.562408    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.565011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.564358    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:11.605219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.605191    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4266b7-openshift-default-dockercfg-nc8ld\""
Oct 30 21:22:11.605985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.605960    4549 generic.go:356] "Generic (PLEG): container finished" podID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" containerID="78a592d97851d272c76d9640f7c8cde341a1f56e5fcafbf7fc14da7824e84911" exitCode=137
Oct 30 21:22:11.616588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.616561    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.616797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.616775    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.616994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.616974    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.617059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.617009    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.617179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.617158    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.617248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.617233    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzqxt\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.617307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.617270    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.617345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.617317    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.617447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.617432    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.620584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.620560    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-d7235c"]
Oct 30 21:22:11.632921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.630056    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-d7235c"]
Oct 30 21:22:11.637953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.637924    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"]
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.724632    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728362    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728406    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728511    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728543    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzqxt\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728580    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728616    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728659    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.728813    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.730096    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.730734    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.731018    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.731787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.731632    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.778926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.731895    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.778926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.733551    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.794857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.786446    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.857984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.849970    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.918864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.892619    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzqxt\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:11.918864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:11.910020    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.082275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.078220    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy\") pod \"e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") " pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:12.262288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.260383    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:12.290125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.267160    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:12.312957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.310030    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.437271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.434884    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk\" (UniqueName: \"kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk\") pod \"594ac4bb-f2fe-409f-be6b-b1392cb817aa\" (UID: \"594ac4bb-f2fe-409f-be6b-b1392cb817aa\") "
Oct 30 21:22:12.442293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.442253    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk" pod "594ac4bb-f2fe-409f-be6b-b1392cb817aa" (UID: "594ac4bb-f2fe-409f-be6b-b1392cb817aa"). InnerVolumeSpecName "kube-api-access-jffjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:12.461890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.460767    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.541675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.541460    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jffjk\" (UniqueName: \"kubernetes.io/projected/594ac4bb-f2fe-409f-be6b-b1392cb817aa-kube-api-access-jffjk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:12.616540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.616512    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:12.616728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.616563    4549 scope.go:122] "RemoveContainer" containerID="78a592d97851d272c76d9640f7c8cde341a1f56e5fcafbf7fc14da7824e84911"
Oct 30 21:22:12.616797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.616773    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7401-d7235c" podUID="872c3995-3fa2-4061-a273-607174ccefc8" containerName="test-curl-7401-d7235c" containerID="cri-o://5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c" gracePeriod=2
Oct 30 21:22:12.629822 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:12.629793    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-d7235c\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="872c3995-3fa2-4061-a273-607174ccefc8" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:12.642938 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:12.642893    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-658e5d\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:12.655986 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:12.655955    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-d7235c\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="872c3995-3fa2-4061-a273-607174ccefc8" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:12.670713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.670627    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"]
Oct 30 21:22:12.671880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.671819    4549 cpu_manager.go:414] "containerMap: removing container" podUID="872c3995-3fa2-4061-a273-607174ccefc8" containerName="test-curl-7401-d7235c"
Oct 30 21:22:12.671880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.671861    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="872c3995-3fa2-4061-a273-607174ccefc8" containerName="test-curl-7401-d7235c"
Oct 30 21:22:12.672068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.672003    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="872c3995-3fa2-4061-a273-607174ccefc8" containerName="test-curl-7401-d7235c"
Oct 30 21:22:12.677425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.677408    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.697073 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:12.690643    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-658e5d\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" pod="opendatahub/test-curl-7401-658e5d"
Oct 30 21:22:12.697073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.695175    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2c7ca6-openshift-default-dockercfg-rw4qx\""
Oct 30 21:22:12.702186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.700610    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"]
Oct 30 21:22:12.703681 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:12.703648    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-d7235c\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="872c3995-3fa2-4061-a273-607174ccefc8" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:12.745924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.745891    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746085    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746117    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746156    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746356    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8c7\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746413    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746526    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746573    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.746776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.746734    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.759319 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:12.759292    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0183f5a1_7bd4_4190_acb0_8dbe84649986.slice/crio-78ced1b09e0f37b7eeb7933e2fdf45e3ee472232b4082b8b1fdb8fa36fb72ad4 WatchSource:0}: Error finding container 78ced1b09e0f37b7eeb7933e2fdf45e3ee472232b4082b8b1fdb8fa36fb72ad4: Status 404 returned error can't find the container with id 78ced1b09e0f37b7eeb7933e2fdf45e3ee472232b4082b8b1fdb8fa36fb72ad4
Oct 30 21:22:12.762466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.762397    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:12.762598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.762486    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:12.762598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.762529    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:12.765344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.765317    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"]
Oct 30 21:22:12.767759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.767737    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.854239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854174    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.854444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854353    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.854531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854506    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.854620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854562    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.854681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854634    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.855029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854771    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8c7\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.855029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854844    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.855029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854909    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.855029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.854936    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.872894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.871298    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.872894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.872269    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.873133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.872926    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.914316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.873342    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.914316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.893573    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.915372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.915310    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:12.925083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.915821    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.943242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:12.943208    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:13.034003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.024744    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:13.101873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.085944    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8c7\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7\") pod \"e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") " pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:13.306276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.302426    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:13.434865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.434222    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="594ac4bb-f2fe-409f-be6b-b1392cb817aa" path="/var/lib/kubelet/pods/594ac4bb-f2fe-409f-be6b-b1392cb817aa/volumes"
Oct 30 21:22:13.481281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.481248    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:13.740071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.695637    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:13.882208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.880559    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" event={"ID":"0183f5a1-7bd4-4190-acb0-8dbe84649986","Type":"ContainerStarted","Data":"b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"}
Oct 30 21:22:13.882208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:13.880636    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" event={"ID":"0183f5a1-7bd4-4190-acb0-8dbe84649986","Type":"ContainerStarted","Data":"78ced1b09e0f37b7eeb7933e2fdf45e3ee472232b4082b8b1fdb8fa36fb72ad4"}
Oct 30 21:22:14.007907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.007756    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:14.288120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.274795    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podStartSLOduration=3.274768525 podStartE2EDuration="3.274768525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:14.00708078 +0000 UTC m=+1316.781480732" watchObservedRunningTime="2026-10-30 21:22:14.274768525 +0000 UTC m=+1317.049168465"
Oct 30 21:22:14.343466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.323788    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:14.343466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.327970    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"]
Oct 30 21:22:14.346990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.346934    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:14.375529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.351409    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:14.375529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.351982    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:14.375529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.360047    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:14.412873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.406910    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:14.863773 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.863745    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:14.901246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.901037    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" event={"ID":"dd146ff3-1f30-4048-9d66-f1eb69c45132","Type":"ContainerStarted","Data":"0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"}
Oct 30 21:22:14.901246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.901115    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" event={"ID":"dd146ff3-1f30-4048-9d66-f1eb69c45132","Type":"ContainerStarted","Data":"d96f521ef2449eb2687db920b998564e5df642fcb574adf7e34fd896fdc4c094"}
Oct 30 21:22:14.905143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.905108    4549 generic.go:356] "Generic (PLEG): container finished" podID="872c3995-3fa2-4061-a273-607174ccefc8" containerID="5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c" exitCode=137
Oct 30 21:22:14.905347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.905151    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:14.905347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.905207    4549 scope.go:122] "RemoveContainer" containerID="5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c"
Oct 30 21:22:14.905672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.905616    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:14.906617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.906602    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:22:14.916285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.916250    4549 scope.go:122] "RemoveContainer" containerID="5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c"
Oct 30 21:22:14.916590 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:14.916567    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c\": container with ID starting with 5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c not found: ID does not exist" containerID="5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c"
Oct 30 21:22:14.916687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.916599    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c"} err="failed to get container status \"5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c\": rpc error: code = NotFound desc = could not find container \"5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c\": container with ID starting with 5ff7b8ab720b4466c7406a1485e557f3563d4013242c4cae246d118511a9385c not found: ID does not exist"
Oct 30 21:22:14.946780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.946722    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podStartSLOduration=2.946708347 podStartE2EDuration="2.946708347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:14.944789579 +0000 UTC m=+1317.719189531" watchObservedRunningTime="2026-10-30 21:22:14.946708347 +0000 UTC m=+1317.721108286"
Oct 30 21:22:14.962448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.962415    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt\" (UniqueName: \"kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt\") pod \"872c3995-3fa2-4061-a273-607174ccefc8\" (UID: \"872c3995-3fa2-4061-a273-607174ccefc8\") "
Oct 30 21:22:14.965076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:14.965044    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt" pod "872c3995-3fa2-4061-a273-607174ccefc8" (UID: "872c3995-3fa2-4061-a273-607174ccefc8"). InnerVolumeSpecName "kube-api-access-nkddt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:15.065423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.065389    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkddt\" (UniqueName: \"kubernetes.io/projected/872c3995-3fa2-4061-a273-607174ccefc8-kube-api-access-nkddt\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:15.252868 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:15.252592    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7401-d7235c\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="872c3995-3fa2-4061-a273-607174ccefc8" pod="opendatahub/test-curl-7401-d7235c"
Oct 30 21:22:15.347012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.346973    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="872c3995-3fa2-4061-a273-607174ccefc8" path="/var/lib/kubelet/pods/872c3995-3fa2-4061-a273-607174ccefc8/volumes"
Oct 30 21:22:15.385845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.385789    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:15.392953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.392917    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:15.480626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.480578    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:15.613306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.613219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:15.915082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.914999    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:15.916004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.915983    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:22:15.924437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:15.924412    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.091275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.091242    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"]
Oct 30 21:22:16.098085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.098061    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.116285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.113354    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w6-908469-openshift-default-dockercfg-bvlc4\""
Oct 30 21:22:16.119241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.118788    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"]
Oct 30 21:22:16.178272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.178151    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.219876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.219790    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.219876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.219866    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.220128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.219910    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.220128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.220016    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cs6h\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.220128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.220065    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.220286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.220180    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.220657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.220425    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.220657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.220512    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.221055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.220729    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322018    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cs6h\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322078    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322189    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322363    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322396    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322474    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322597    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322645    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322687    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.322946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.322903    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.323342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.323316    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.323980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.323956    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.324090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.323984    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.324313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.324294    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.325360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.325336    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.325481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.325460    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.342353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.342325    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.342495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.342430    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cs6h\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h\") pod \"e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") " pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.354519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.354488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.441905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.435345    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:16.529855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.526560    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.687944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.679109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.889199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.885417    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"]
Oct 30 21:22:16.933665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.929171    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:16.933665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.930415    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"]
Oct 30 21:22:16.951320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:16.951274    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-75c17d-openshift-default-dockercfg-48vwh\""
Oct 30 21:22:17.171940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.168703    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:17.491380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.482924    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"]
Oct 30 21:22:17.593254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.593201    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" containerID="cri-o://b9326704c2e6660318ce9f1ef76978e94dcfe4d63ca5e6c1bb9c8a29f00d0018" gracePeriod=30
Oct 30 21:22:17.741143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.741114    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.742980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.742611    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.743194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.743169    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5klqs\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.745440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.745418    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.745665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.745648    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.746084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.746064    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.832903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.825335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:17.843694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.841759    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.843694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.841878    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.843694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.842091    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.910314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.910263    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:17.910471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.910364    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:17.910471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.910403    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:17.921520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.921191    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"]
Oct 30 21:22:17.921520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.921249    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"]
Oct 30 21:22:17.921520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.921433    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy" containerID="cri-o://a7e0cf91ce49a66f960d92301c9b62ff0d3e6f7781a5e3cdb08e44393c1d0f69" gracePeriod=30
Oct 30 21:22:17.922083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.921663    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy" containerID="cri-o://e90d1d052b10ec3919ab233bb504fd14afadfa3818d6f46c920087e9d2f29b3f" gracePeriod=30
Oct 30 21:22:17.924903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.924745    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"]
Oct 30 21:22:17.932090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.932063    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"]
Oct 30 21:22:17.932295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.932264    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy" containerID="cri-o://fa33cbeadd7b7d0a21e7e369f7ba82e88fb4ef868af26762a19e55f9a36560d7" gracePeriod=30
Oct 30 21:22:17.936252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.936232    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"]
Oct 30 21:22:17.936454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.936429    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" containerID="cri-o://b96cbda3c5a1db6577174d82f68e51a431a9ad250103b4886df3e66f556f664d" gracePeriod=30
Oct 30 21:22:17.940650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.940627    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"]
Oct 30 21:22:17.940821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.940801    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy" containerID="cri-o://d8ea42a4defa9da41a3afcb4935765d7c9760f045f15cdad1cdb6748665a8829" gracePeriod=30
Oct 30 21:22:17.943513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.943484    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.943684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.943563    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.943782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.943699    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.943892    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944001    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944201    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944223    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5klqs\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944259    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944276    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.944545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944506    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.945009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.944979    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.945457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.945357    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.945946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.945916    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.946869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.946845    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.947234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.947215    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.947737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.947715    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"]
Oct 30 21:22:17.947898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.947875    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.947986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.947953    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy" containerID="cri-o://316cc882c8cb7e6feee86131cf91a5fdcf401fda7325a05ca991dc2d02eeb4ee" gracePeriod=30
Oct 30 21:22:17.955237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.955207    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"]
Oct 30 21:22:17.955369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.955350    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy" containerID="cri-o://a452281bf5a452da8ad1f21969011ee97f99dd4712253412a0e2da00030e5912" gracePeriod=30
Oct 30 21:22:17.959215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.959194    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"]
Oct 30 21:22:17.959377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.959354    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy" containerID="cri-o://cb6ac08dd4dda2ffad35159ec641ee6626585dfb93cd248185998a4bdf34a7d8" gracePeriod=30
Oct 30 21:22:17.968501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.968474    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.970573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.970541    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5klqs\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs\") pod \"e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") " pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:17.970727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.970611    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"]
Oct 30 21:22:17.975260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.975233    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:17.977839 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.977801    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" event={"ID":"eba83462-ebf4-472f-8e5c-cb7b42c25321","Type":"ContainerStarted","Data":"8f89c42350da9a3d6315da4d334dea71e7446a5fffa08f05625311980d5d99ac"}
Oct 30 21:22:17.978032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:17.978010    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy" containerID="cri-o://3cabc901e600f447922fd6553310a555903ca5562abd445d94e0212d5d24573e" gracePeriod=30
Oct 30 21:22:18.235333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.235302    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-75c17d-openshift-default-dockercfg-48vwh\""
Oct 30 21:22:18.242068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.242038    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:18.337961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.337930    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-2fb6b6"]
Oct 30 21:22:18.355133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.345536    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:18.368731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.368548    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxp2m\""
Oct 30 21:22:18.373308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.373283    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-2fb6b6"]
Oct 30 21:22:18.392792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.392761    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:18.459379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.459340    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wpl\" (UniqueName: \"kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl\") pod \"test-curl-7404-2fb6b6\" (UID: \"8626b713-2e11-4bd7-a3b3-cc98f75c44a5\") " pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:18.560634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.560606    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:18.561463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.561433    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wpl\" (UniqueName: \"kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl\") pod \"test-curl-7404-2fb6b6\" (UID: \"8626b713-2e11-4bd7-a3b3-cc98f75c44a5\") " pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:18.589463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.589372    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"]
Oct 30 21:22:18.737278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.737175    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wpl\" (UniqueName: \"kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl\") pod \"test-curl-7404-2fb6b6\" (UID: \"8626b713-2e11-4bd7-a3b3-cc98f75c44a5\") " pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:18.792056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.782435    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:18.792056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.782537    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:18.792056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:18.782605    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:19.036853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.025862    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:19.158565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.158340    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" event={"ID":"f06ed5e1-a07c-43e7-8387-db7a68cc97a7","Type":"ContainerStarted","Data":"ad1133e4ebb15c9d647f1546480e2ef0317219fe95d121ad6af10b73cf0d7a5c"}
Oct 30 21:22:19.168023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.165137    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" event={"ID":"eba83462-ebf4-472f-8e5c-cb7b42c25321","Type":"ContainerStarted","Data":"1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"}
Oct 30 21:22:19.223280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.222906    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podStartSLOduration=3.222885647 podStartE2EDuration="3.222885647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:19.216313604 +0000 UTC m=+1321.990713545" watchObservedRunningTime="2026-10-30 21:22:19.222885647 +0000 UTC m=+1321.997285589"
Oct 30 21:22:19.445312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.445279    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:19.453820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.453792    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:19.553855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.547877    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:19.633354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:19.633314    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-2fb6b6"]
Oct 30 21:22:20.012675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.012638    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:20.148276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.148239    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:20.175554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.175514    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-2fb6b6" event={"ID":"8626b713-2e11-4bd7-a3b3-cc98f75c44a5","Type":"ContainerStarted","Data":"c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d"}
Oct 30 21:22:20.175751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.175570    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-2fb6b6" event={"ID":"8626b713-2e11-4bd7-a3b3-cc98f75c44a5","Type":"ContainerStarted","Data":"ce038d093d4be4d4318562956ebd7406224fc059a75974a98df960dc291ca688"}
Oct 30 21:22:20.177721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.177683    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" event={"ID":"f06ed5e1-a07c-43e7-8387-db7a68cc97a7","Type":"ContainerStarted","Data":"922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"}
Oct 30 21:22:20.177923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.177903    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:20.179087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.179065    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:22:20.360945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.360063    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:20.619917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.619791    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:20.706201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.706124    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7404-2fb6b6" podStartSLOduration=2.706097928 podStartE2EDuration="2.706097928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:20.216560162 +0000 UTC m=+1322.990960102" watchObservedRunningTime="2026-10-30 21:22:20.706097928 +0000 UTC m=+1323.480497883"
Oct 30 21:22:20.789323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:20.758294    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podStartSLOduration=4.758269843 podStartE2EDuration="4.758269843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:20.359892474 +0000 UTC m=+1323.134292414" watchObservedRunningTime="2026-10-30 21:22:20.758269843 +0000 UTC m=+1323.532669783"
Oct 30 21:22:21.186424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.186393    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:21.187334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.187311    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:22:21.644540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.644509    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-2fb6b6"]
Oct 30 21:22:21.653759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.653731    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-2fb6b6"]
Oct 30 21:22:21.710580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.710540    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:21.711194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.711146    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:22:21.711295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.711216    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:22:21.855883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:21.846226    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:22.005101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:22.005000    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:22.281908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:22.279923    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7404-2fb6b6" podUID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" containerName="test-curl-7404-2fb6b6" containerID="cri-o://c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d" gracePeriod=2
Oct 30 21:22:22.321853 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:22.296075    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7404-2fb6b6\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:23.260359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.202179    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:22:23.260359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.202241    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:22:23.318964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.318935    4549 generic.go:356] "Generic (PLEG): container finished" podID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerID="a7e0cf91ce49a66f960d92301c9b62ff0d3e6f7781a5e3cdb08e44393c1d0f69" exitCode=0
Oct 30 21:22:23.319094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.319042    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" event={"ID":"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b","Type":"ContainerDied","Data":"a7e0cf91ce49a66f960d92301c9b62ff0d3e6f7781a5e3cdb08e44393c1d0f69"}
Oct 30 21:22:23.324879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.322792    4549 generic.go:356] "Generic (PLEG): container finished" podID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerID="fa33cbeadd7b7d0a21e7e369f7ba82e88fb4ef868af26762a19e55f9a36560d7" exitCode=0
Oct 30 21:22:23.324879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.322906    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" event={"ID":"060dbc88-01cf-4f36-8f18-2cf414a32c2a","Type":"ContainerDied","Data":"fa33cbeadd7b7d0a21e7e369f7ba82e88fb4ef868af26762a19e55f9a36560d7"}
Oct 30 21:22:23.325133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.325096    4549 generic.go:356] "Generic (PLEG): container finished" podID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerID="b96cbda3c5a1db6577174d82f68e51a431a9ad250103b4886df3e66f556f664d" exitCode=0
Oct 30 21:22:23.325216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.325190    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" event={"ID":"203335d3-dfdf-40dc-8936-db10cb422b4e","Type":"ContainerDied","Data":"b96cbda3c5a1db6577174d82f68e51a431a9ad250103b4886df3e66f556f664d"}
Oct 30 21:22:23.339013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.327289    4549 generic.go:356] "Generic (PLEG): container finished" podID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerID="a452281bf5a452da8ad1f21969011ee97f99dd4712253412a0e2da00030e5912" exitCode=0
Oct 30 21:22:23.339013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.327362    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" event={"ID":"040183c8-21bd-44d3-bde2-bc99b6e3bc9f","Type":"ContainerDied","Data":"a452281bf5a452da8ad1f21969011ee97f99dd4712253412a0e2da00030e5912"}
Oct 30 21:22:23.339013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.329612    4549 generic.go:356] "Generic (PLEG): container finished" podID="b1baa751-388b-42f2-829c-7dfc01f41627" containerID="cb6ac08dd4dda2ffad35159ec641ee6626585dfb93cd248185998a4bdf34a7d8" exitCode=0
Oct 30 21:22:23.339013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.329696    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" event={"ID":"b1baa751-388b-42f2-829c-7dfc01f41627","Type":"ContainerDied","Data":"cb6ac08dd4dda2ffad35159ec641ee6626585dfb93cd248185998a4bdf34a7d8"}
Oct 30 21:22:23.339013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.333283    4549 generic.go:356] "Generic (PLEG): container finished" podID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerID="e90d1d052b10ec3919ab233bb504fd14afadfa3818d6f46c920087e9d2f29b3f" exitCode=0
Oct 30 21:22:23.339013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.333374    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" event={"ID":"0b8e0a11-ee33-4e51-826d-8636f62d79b1","Type":"ContainerDied","Data":"e90d1d052b10ec3919ab233bb504fd14afadfa3818d6f46c920087e9d2f29b3f"}
Oct 30 21:22:23.343234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.343211    4549 generic.go:356] "Generic (PLEG): container finished" podID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerID="b9326704c2e6660318ce9f1ef76978e94dcfe4d63ca5e6c1bb9c8a29f00d0018" exitCode=0
Oct 30 21:22:23.345408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.345360    4549 generic.go:356] "Generic (PLEG): container finished" podID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerID="3cabc901e600f447922fd6553310a555903ca5562abd445d94e0212d5d24573e" exitCode=0
Oct 30 21:22:23.351851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.347855    4549 generic.go:356] "Generic (PLEG): container finished" podID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerID="316cc882c8cb7e6feee86131cf91a5fdcf401fda7325a05ca991dc2d02eeb4ee" exitCode=0
Oct 30 21:22:23.351851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.350104    4549 generic.go:356] "Generic (PLEG): container finished" podID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerID="d8ea42a4defa9da41a3afcb4935765d7c9760f045f15cdad1cdb6748665a8829" exitCode=0
Oct 30 21:22:23.387351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.387203    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" event={"ID":"6557f057-f9a4-4e7d-93a8-106f9d6248a3","Type":"ContainerDied","Data":"b9326704c2e6660318ce9f1ef76978e94dcfe4d63ca5e6c1bb9c8a29f00d0018"}
Oct 30 21:22:23.387351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.387257    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr" event={"ID":"6557f057-f9a4-4e7d-93a8-106f9d6248a3","Type":"ContainerDied","Data":"6ef772087a3de8342adb842753182080fe53057545c68eada72ea11ac1e2b30f"}
Oct 30 21:22:23.387351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.387274    4549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ef772087a3de8342adb842753182080fe53057545c68eada72ea11ac1e2b30f"
Oct 30 21:22:23.387351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.387282    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" event={"ID":"2ba9d8ef-3105-4591-af9e-9fee8a9c577d","Type":"ContainerDied","Data":"3cabc901e600f447922fd6553310a555903ca5562abd445d94e0212d5d24573e"}
Oct 30 21:22:23.387351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.387300    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" event={"ID":"9f4b1567-6039-4afc-8d7d-72ff11a0fa69","Type":"ContainerDied","Data":"316cc882c8cb7e6feee86131cf91a5fdcf401fda7325a05ca991dc2d02eeb4ee"}
Oct 30 21:22:23.387351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.387316    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" event={"ID":"c92e1b1c-a803-4186-b935-4f0cc43b3d85","Type":"ContainerDied","Data":"d8ea42a4defa9da41a3afcb4935765d7c9760f045f15cdad1cdb6748665a8829"}
Oct 30 21:22:23.447851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.445030    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:22:23.521566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.521020    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.521791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.521510    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.526882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.521943    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.526882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.526401    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527133    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527214    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527325    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527363    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527401    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527486    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.528210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.527666    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data\") pod \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\" (UID: \"6557f057-f9a4-4e7d-93a8-106f9d6248a3\") "
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529254    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529498    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529759    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529884    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529905    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529918    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.530114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.529930    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.535064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.535038    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.546908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.546795    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:22:23.552466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.552438    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "kube-api-access-wcfsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.552733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.552713    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:22:23.553171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.553142    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.553567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.553547    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token" pod "6557f057-f9a4-4e7d-93a8-106f9d6248a3" (UID: "6557f057-f9a4-4e7d-93a8-106f9d6248a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.631897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.630791    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.631996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.631912    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.631996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.631973    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.632121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.632027    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.632645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.632523    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.632952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.632928    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.633098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633074    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.633241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633172    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633207    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633243    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633300    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.633436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633412    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.633493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633438    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.633493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633464    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633507    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633565    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.633698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633602    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633627    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.633698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633646    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.633863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633792    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4\") pod \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\" (UID: \"060dbc88-01cf-4f36-8f18-2cf414a32c2a\") "
Oct 30 21:22:23.633863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633848    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.633973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633855    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.633973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633921    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket\") pod \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\" (UID: \"0b8e0a11-ee33-4e51-826d-8636f62d79b1\") "
Oct 30 21:22:23.634113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.633962    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.634537    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635054    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635127    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635453    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635623    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635648    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635662    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635681    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635701    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635715    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635729    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635743    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635759    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635776    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635789    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635801    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635812    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635878    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcfsf\" (UniqueName: \"kubernetes.io/projected/6557f057-f9a4-4e7d-93a8-106f9d6248a3-kube-api-access-wcfsf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.635894    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.638009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.637952    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.640962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.640550    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.642190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.641500    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.642190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.641764    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.642190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.642159    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "kube-api-access-dfr9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.645513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.644135    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.646085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.646063    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4" pod "060dbc88-01cf-4f36-8f18-2cf414a32c2a" (UID: "060dbc88-01cf-4f36-8f18-2cf414a32c2a"). InnerVolumeSpecName "kube-api-access-77tq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.646333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.646303    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token" pod "0b8e0a11-ee33-4e51-826d-8636f62d79b1" (UID: "0b8e0a11-ee33-4e51-826d-8636f62d79b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.648740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.648648    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:22:23.650958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.650935    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:22:23.653395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.653313    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:22:23.666645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.666617    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:22:23.698450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.698426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:23.739638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.739604    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.739959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.739940    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.740096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740083    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.740217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740200    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.740327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740314    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.740483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740471    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.740605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740593    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.740746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740733    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.740901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740887    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.741032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741008    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.741143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741130    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.741240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741228    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.741391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741364    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.741502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741488    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.741629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741617    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.741741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741724    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.739948    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.740216    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741796    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741864    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741921    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.741972    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742009    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742129    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742164    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742201    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742267    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742296    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742335    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742372    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742430    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742482    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742604    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742634    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs\") pod \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\" (UID: \"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.742745    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket\") pod \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\" (UID: \"2ba9d8ef-3105-4591-af9e-9fee8a9c577d\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.743007    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.743043    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.743084    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert\") pod \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\" (UID: \"9f4b1567-6039-4afc-8d7d-72ff11a0fa69\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.743127    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo\") pod \"203335d3-dfdf-40dc-8936-db10cb422b4e\" (UID: \"203335d3-dfdf-40dc-8936-db10cb422b4e\") "
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744270    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744296    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744313    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744327    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744341    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfr9t\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-kube-api-access-dfr9t\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744356    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744371    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744386    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744401    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77tq4\" (UniqueName: \"kubernetes.io/projected/060dbc88-01cf-4f36-8f18-2cf414a32c2a-kube-api-access-77tq4\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744416    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b8e0a11-ee33-4e51-826d-8636f62d79b1-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744429    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b8e0a11-ee33-4e51-826d-8636f62d79b1-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744709    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.744815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744759    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.744944    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.745769    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.746015    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.746363    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.746547    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.746749    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.749653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.749171    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.750348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.750323    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.750426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.750382    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.752845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.750864    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.752845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.751242    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.752845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.751506    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.752845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.752460    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.753912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.752887    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.753912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.753180    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.753912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.753421    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.756560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.756385    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.756560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.756396    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.756560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.756495    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "kube-api-access-ths4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757233    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757307    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757320    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757344    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757360    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757404    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z" pod "9f4b1567-6039-4afc-8d7d-72ff11a0fa69" (UID: "9f4b1567-6039-4afc-8d7d-72ff11a0fa69"). InnerVolumeSpecName "kube-api-access-4m72z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.759119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.757541    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo" pod "203335d3-dfdf-40dc-8936-db10cb422b4e" (UID: "203335d3-dfdf-40dc-8936-db10cb422b4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.775240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.759968    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.775240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.759992    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "kube-api-access-r2r7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.775240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.760011    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4" pod "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" (UID: "fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b"). InnerVolumeSpecName "kube-api-access-g4gd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.775240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.760032    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.775240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.771983    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token" pod "2ba9d8ef-3105-4591-af9e-9fee8a9c577d" (UID: "2ba9d8ef-3105-4591-af9e-9fee8a9c577d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.779550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.779525    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:22:23.793168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.781666    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:22:23.797118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.797093    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847055    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847145    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847302    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847434    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847466    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847503    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847561    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847612    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847644    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847686    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847735    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847762    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847804    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.847879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847881    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.848727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.847936    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.848727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.848053    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket\") pod \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\" (UID: \"c92e1b1c-a803-4186-b935-4f0cc43b3d85\") "
Oct 30 21:22:23.848727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.848089    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.848727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.848122    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket\") pod \"b1baa751-388b-42f2-829c-7dfc01f41627\" (UID: \"b1baa751-388b-42f2-829c-7dfc01f41627\") "
Oct 30 21:22:23.849122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849101    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849132    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849146    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849160    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849173    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849191    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849205    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849217    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849230    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849243    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849259    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ths4d\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-kube-api-access-ths4d\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849273    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849287    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849300    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849314    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849329    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849341    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203335d3-dfdf-40dc-8936-db10cb422b4e-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849354    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849367    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849379    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849392    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2r7l\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-kube-api-access-r2r7l\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849407    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849420    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m72z\" (UniqueName: \"kubernetes.io/projected/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-kube-api-access-4m72z\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849434    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849449    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f4b1567-6039-4afc-8d7d-72ff11a0fa69-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849463    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849476    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203335d3-dfdf-40dc-8936-db10cb422b4e-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849489    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849504    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849517    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849531    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849544    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba9d8ef-3105-4591-af9e-9fee8a9c577d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.849572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.849557    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4gd4\" (UniqueName: \"kubernetes.io/projected/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b-kube-api-access-g4gd4\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.851716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.851018    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.851716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.851292    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.851716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.851331    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.851716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.851599    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.855349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.855267    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.855465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.855384    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.856062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.855803    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.856756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.856573    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.856756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.856601    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "kube-api-access-zmnpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.856922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.856843    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.857193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.857158    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.857324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.857213    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.865048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.858211    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.865048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.859647    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.865048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.864385    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.882878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.873234    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.882878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.873279    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy" pod "b1baa751-388b-42f2-829c-7dfc01f41627" (UID: "b1baa751-388b-42f2-829c-7dfc01f41627"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.882878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.875284    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w" pod "c92e1b1c-a803-4186-b935-4f0cc43b3d85" (UID: "c92e1b1c-a803-4186-b935-4f0cc43b3d85"). InnerVolumeSpecName "kube-api-access-8587w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.950758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.950646    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.950758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.950751    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.950935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.950798    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.950999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.950940    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.951215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.951180    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.951276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.951237    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.951390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.951342    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.951488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.951406    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.951574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.951490    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert\") pod \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\" (UID: \"040183c8-21bd-44d3-bde2-bc99b6e3bc9f\") "
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952430    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952459    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952473    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952485    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952497    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952509    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1baa751-388b-42f2-829c-7dfc01f41627-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952522    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952536    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952553    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952566    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1baa751-388b-42f2-829c-7dfc01f41627-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952580    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmnpb\" (UniqueName: \"kubernetes.io/projected/b1baa751-388b-42f2-829c-7dfc01f41627-kube-api-access-zmnpb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952593    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952606    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952619    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952632    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8587w\" (UniqueName: \"kubernetes.io/projected/c92e1b1c-a803-4186-b935-4f0cc43b3d85-kube-api-access-8587w\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952644    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952657    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92e1b1c-a803-4186-b935-4f0cc43b3d85-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.952669    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1baa751-388b-42f2-829c-7dfc01f41627-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:23.953455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.953093    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:23.955866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.955675    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.955994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.955885    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.956369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.956317    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.956458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.956386    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:23.958216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.958182    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:23.958622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.958581    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.960279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.960253    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "kube-api-access-2dzbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:23.960419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:23.960394    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy" pod "040183c8-21bd-44d3-bde2-bc99b6e3bc9f" (UID: "040183c8-21bd-44d3-bde2-bc99b6e3bc9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:24.054273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.054238    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.054579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.054557    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.054760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.054743    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.054950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.054934    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.055091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.055059    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.055224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.055210    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.055355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.055341    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.055502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.055469    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dzbg\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-kube-api-access-2dzbg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.055619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.055606    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040183c8-21bd-44d3-bde2-bc99b6e3bc9f-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.087581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.087550    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:24.447853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.444619    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq" event={"ID":"c92e1b1c-a803-4186-b935-4f0cc43b3d85","Type":"ContainerDied","Data":"24d3761846b26acebd296e407669fcf5912af4a0a7ee861ee4fd4df9a0ff1cd1"}
Oct 30 21:22:24.447853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.444693    4549 scope.go:122] "RemoveContainer" containerID="d8ea42a4defa9da41a3afcb4935765d7c9760f045f15cdad1cdb6748665a8829"
Oct 30 21:22:24.447853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.444909    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"
Oct 30 21:22:24.477885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.477846    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t" event={"ID":"fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b","Type":"ContainerDied","Data":"acf1185feeed2616a4d7e0ff48addaac84256ca604241e4f8be5b2df4548997e"}
Oct 30 21:22:24.478267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.478246    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"
Oct 30 21:22:24.499645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.492031    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck" event={"ID":"060dbc88-01cf-4f36-8f18-2cf414a32c2a","Type":"ContainerDied","Data":"a3174819611e66682b0cf015fe028a2a523256e022d47e75db962c784333f5a4"}
Oct 30 21:22:24.499645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.492151    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"
Oct 30 21:22:24.545997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.536195    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"
Oct 30 21:22:24.545997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.537022    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj" event={"ID":"203335d3-dfdf-40dc-8936-db10cb422b4e","Type":"ContainerDied","Data":"4d84900d56cf1a7d2e3ecda6de1dabeb2a2279b3657c96379c7641cbb22aafdc"}
Oct 30 21:22:24.576978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.576850    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn" event={"ID":"040183c8-21bd-44d3-bde2-bc99b6e3bc9f","Type":"ContainerDied","Data":"e154f9a4d1961c5c019117e21949a2a999c23a0cd5b02429992ede27467f7402"}
Oct 30 21:22:24.577106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.577014    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"
Oct 30 21:22:24.581859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.581169    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426" event={"ID":"b1baa751-388b-42f2-829c-7dfc01f41627","Type":"ContainerDied","Data":"18987d188c84c2db947f8a5ec3cf7a185f9578fd3035a15f828f6df0a3800751"}
Oct 30 21:22:24.581859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.581287    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"
Oct 30 21:22:24.591333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.591300    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6" event={"ID":"0b8e0a11-ee33-4e51-826d-8636f62d79b1","Type":"ContainerDied","Data":"5015f1feaa1f309f124ef4318873a65d995b97e5db7d7fdb1bae892dacf141f5"}
Oct 30 21:22:24.591509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.591417    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"
Oct 30 21:22:24.599155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.599128    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"
Oct 30 21:22:24.599887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.599769    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf" event={"ID":"2ba9d8ef-3105-4591-af9e-9fee8a9c577d","Type":"ContainerDied","Data":"f14a4ab065ca96a8ee29fd05e8df8bec0b62e90dfe64819259d653f999c66a91"}
Oct 30 21:22:24.602080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.602059    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"
Oct 30 21:22:24.603540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.602673    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9" event={"ID":"9f4b1567-6039-4afc-8d7d-72ff11a0fa69","Type":"ContainerDied","Data":"a22166f0de9cc9e221d26b237a2761658808c2721893a5bf2b446389a8cfec7f"}
Oct 30 21:22:24.603540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.602729    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"
Oct 30 21:22:24.625247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.625226    4549 scope.go:122] "RemoveContainer" containerID="a7e0cf91ce49a66f960d92301c9b62ff0d3e6f7781a5e3cdb08e44393c1d0f69"
Oct 30 21:22:24.647004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.646984    4549 scope.go:122] "RemoveContainer" containerID="fa33cbeadd7b7d0a21e7e369f7ba82e88fb4ef868af26762a19e55f9a36560d7"
Oct 30 21:22:24.660876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.660856    4549 scope.go:122] "RemoveContainer" containerID="b96cbda3c5a1db6577174d82f68e51a431a9ad250103b4886df3e66f556f664d"
Oct 30 21:22:24.673000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.672934    4549 scope.go:122] "RemoveContainer" containerID="a452281bf5a452da8ad1f21969011ee97f99dd4712253412a0e2da00030e5912"
Oct 30 21:22:24.684071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.684045    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"]
Oct 30 21:22:24.686283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.686254    4549 scope.go:122] "RemoveContainer" containerID="cb6ac08dd4dda2ffad35159ec641ee6626585dfb93cd248185998a4bdf34a7d8"
Oct 30 21:22:24.696960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.696928    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck"]
Oct 30 21:22:24.697752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.697733    4549 scope.go:122] "RemoveContainer" containerID="e90d1d052b10ec3919ab233bb504fd14afadfa3818d6f46c920087e9d2f29b3f"
Oct 30 21:22:24.711359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.711332    4549 scope.go:122] "RemoveContainer" containerID="3cabc901e600f447922fd6553310a555903ca5562abd445d94e0212d5d24573e"
Oct 30 21:22:24.736592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.736554    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"]
Oct 30 21:22:24.758615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.758590    4549 scope.go:122] "RemoveContainer" containerID="316cc882c8cb7e6feee86131cf91a5fdcf401fda7325a05ca991dc2d02eeb4ee"
Oct 30 21:22:24.760262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.760233    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426"]
Oct 30 21:22:24.781461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.781433    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"]
Oct 30 21:22:24.786934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.786910    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq"]
Oct 30 21:22:24.803317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.803289    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:24.821085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.821041    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl\" (UniqueName: \"kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl\") pod \"8626b713-2e11-4bd7-a3b3-cc98f75c44a5\" (UID: \"8626b713-2e11-4bd7-a3b3-cc98f75c44a5\") "
Oct 30 21:22:24.822453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.822431    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"]
Oct 30 21:22:24.823400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.823379    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl" pod "8626b713-2e11-4bd7-a3b3-cc98f75c44a5" (UID: "8626b713-2e11-4bd7-a3b3-cc98f75c44a5"). InnerVolumeSpecName "kube-api-access-t6wpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:24.830158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.830135    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn"]
Oct 30 21:22:24.865091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.865062    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"]
Oct 30 21:22:24.870274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.870250    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9"]
Oct 30 21:22:24.904403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.904370    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"]
Oct 30 21:22:24.908431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.908408    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t"]
Oct 30 21:22:24.924061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.924037    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6wpl\" (UniqueName: \"kubernetes.io/projected/8626b713-2e11-4bd7-a3b3-cc98f75c44a5-kube-api-access-t6wpl\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:24.980322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.980249    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"]
Oct 30 21:22:24.984603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:24.984578    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj"]
Oct 30 21:22:25.022438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.022406    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"]
Oct 30 21:22:25.027679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.027654    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr"]
Oct 30 21:22:25.058565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.058532    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"]
Oct 30 21:22:25.068119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.068093    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf"]
Oct 30 21:22:25.105173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.105140    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"]
Oct 30 21:22:25.114409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.114381    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6"]
Oct 30 21:22:25.224671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.224636    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"]
Oct 30 21:22:25.225427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225404    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy"
Oct 30 21:22:25.225427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225425    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225436    4549 cpu_manager.go:414] "containerMap: removing container" podUID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" containerName="test-curl-7404-2fb6b6"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225441    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" containerName="test-curl-7404-2fb6b6"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225453    4549 cpu_manager.go:414] "containerMap: removing container" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225458    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225467    4549 cpu_manager.go:414] "containerMap: removing container" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225471    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225481    4549 cpu_manager.go:414] "containerMap: removing container" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225485    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225493    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225497    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225507    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225511    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225520    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225524    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225551    4549 cpu_manager.go:414] "containerMap: removing container" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225555    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225560    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225564    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225574    4549 cpu_manager.go:414] "containerMap: removing container" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy"
Oct 30 21:22:25.225606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225578    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225671    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225681    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225687    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" containerName="test-curl-7404-2fb6b6"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225693    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225698    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225708    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225714    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225719    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225725    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225733    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" containerName="istio-proxy"
Oct 30 21:22:25.226396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.225738    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" containerName="istio-proxy"
Oct 30 21:22:25.229470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.229454    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.250002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.249977    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-59fb17-openshift-default-dockercfg-v8czt\""
Oct 30 21:22:25.263094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.263065    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"]
Oct 30 21:22:25.268946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.268919    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"]
Oct 30 21:22:25.273485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.273462    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.296719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.296691    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"]
Oct 30 21:22:25.332055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332018    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332080    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lwrp\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332197    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332228    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332266    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.332482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332334    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.332482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332361    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.332482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332411    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.332482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332460    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332494    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332525    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.332797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.332606    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.333555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.333526    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.333822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.333803    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmmq\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.333960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.333884    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.333960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.333942    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.334067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.334051    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.334119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.334103    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.353849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.351998    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="040183c8-21bd-44d3-bde2-bc99b6e3bc9f" path="/var/lib/kubelet/pods/040183c8-21bd-44d3-bde2-bc99b6e3bc9f/volumes"
Oct 30 21:22:25.353849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.352788    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="060dbc88-01cf-4f36-8f18-2cf414a32c2a" path="/var/lib/kubelet/pods/060dbc88-01cf-4f36-8f18-2cf414a32c2a/volumes"
Oct 30 21:22:25.353849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.353455    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b8e0a11-ee33-4e51-826d-8636f62d79b1" path="/var/lib/kubelet/pods/0b8e0a11-ee33-4e51-826d-8636f62d79b1/volumes"
Oct 30 21:22:25.354609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.354579    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="203335d3-dfdf-40dc-8936-db10cb422b4e" path="/var/lib/kubelet/pods/203335d3-dfdf-40dc-8936-db10cb422b4e/volumes"
Oct 30 21:22:25.355294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.355274    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ba9d8ef-3105-4591-af9e-9fee8a9c577d" path="/var/lib/kubelet/pods/2ba9d8ef-3105-4591-af9e-9fee8a9c577d/volumes"
Oct 30 21:22:25.355952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.355933    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6557f057-f9a4-4e7d-93a8-106f9d6248a3" path="/var/lib/kubelet/pods/6557f057-f9a4-4e7d-93a8-106f9d6248a3/volumes"
Oct 30 21:22:25.359876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.356874    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" path="/var/lib/kubelet/pods/8626b713-2e11-4bd7-a3b3-cc98f75c44a5/volumes"
Oct 30 21:22:25.359876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.357520    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f4b1567-6039-4afc-8d7d-72ff11a0fa69" path="/var/lib/kubelet/pods/9f4b1567-6039-4afc-8d7d-72ff11a0fa69/volumes"
Oct 30 21:22:25.359876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.358176    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1baa751-388b-42f2-829c-7dfc01f41627" path="/var/lib/kubelet/pods/b1baa751-388b-42f2-829c-7dfc01f41627/volumes"
Oct 30 21:22:25.359876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.359258    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c92e1b1c-a803-4186-b935-4f0cc43b3d85" path="/var/lib/kubelet/pods/c92e1b1c-a803-4186-b935-4f0cc43b3d85/volumes"
Oct 30 21:22:25.360119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.359918    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b" path="/var/lib/kubelet/pods/fdbe5a2b-bed0-4b5d-b46b-f0a0fced053b/volumes"
Oct 30 21:22:25.420981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.420951    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:25.435647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435620    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.435839 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435683    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.435839 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435749    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmmq\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.435839 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435777    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435841    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435908    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.435955    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436074    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436111    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lwrp\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436149    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436155    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436176    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436319    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436370    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436412    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436508    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436518    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436533    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436549    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436605    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436681    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.436940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436854    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.436940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436803    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.437045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436967    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.437120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.436957    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.438156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.438111    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.438289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.438213    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.438734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.438711    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.439608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.439586    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.439811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.439789    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.439952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.439918    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.440095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.440078    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.465283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.465256    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lwrp\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.469647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.469618    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmmq\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.472879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.472853    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.474733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.474706    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token\") pod \"e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") " pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.533270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.533187    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"]
Oct 30 21:22:25.540635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.540530    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:25.550604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.550574    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:25.567762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.567743    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-75c17d\""
Oct 30 21:22:25.579987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.579959    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"]
Oct 30 21:22:25.590028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.590000    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:25.617433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.616980    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"]
Oct 30 21:22:25.633369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.633256    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:25.710013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.687111    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-75c17d-dockercfg-9lrq6\""
Oct 30 21:22:25.791036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.712141    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"]
Oct 30 21:22:25.791036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.790347    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:25.791036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.790664    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvcx\" (UniqueName: \"kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx\") pod \"payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:25.925303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.925063    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hw6v\" (UniqueName: \"kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v\") pod \"payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:25.925303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:25.925105    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:26.072877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.066292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.084706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.084641    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvcx\" (UniqueName: \"kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx\") pod \"payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:26.085239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.085191    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hw6v\" (UniqueName: \"kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v\") pod \"payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:26.085406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.085390    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:26.085558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.085544    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:26.086677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.086657    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:26.093423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.090554    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:26.133353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.114294    4549 generic.go:356] "Generic (PLEG): container finished" podID="8626b713-2e11-4bd7-a3b3-cc98f75c44a5" containerID="c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d" exitCode=137
Oct 30 21:22:26.133353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.114460    4549 scope.go:122] "RemoveContainer" containerID="c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d"
Oct 30 21:22:26.133353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.114611    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-2fb6b6"
Oct 30 21:22:26.133353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.131597    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvcx\" (UniqueName: \"kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx\") pod \"payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:26.142088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.142016    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hw6v\" (UniqueName: \"kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v\") pod \"payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:26.159744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.159717    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:26.175120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.175084    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:26.180783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.180706    4549 scope.go:122] "RemoveContainer" containerID="c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d"
Oct 30 21:22:26.181180 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:26.181143    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d\": container with ID starting with c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d not found: ID does not exist" containerID="c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d"
Oct 30 21:22:26.181268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.181205    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d"} err="failed to get container status \"c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d\": rpc error: code = NotFound desc = could not find container \"c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d\": container with ID starting with c663558730d63a7f553af522c1adc0384d61210d151d0b28b40153f3f06a5a5d not found: ID does not exist"
Oct 30 21:22:26.401232 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:26.401197    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddf7906_05f9_40d5_a863_28b79fedcf3f.slice/crio-adaba52dbd367ecc257d7b932a4d2ce42d33c0dea7e3748e7a21d69fb0027b25 WatchSource:0}: Error finding container adaba52dbd367ecc257d7b932a4d2ce42d33c0dea7e3748e7a21d69fb0027b25: Status 404 returned error can't find the container with id adaba52dbd367ecc257d7b932a4d2ce42d33c0dea7e3748e7a21d69fb0027b25
Oct 30 21:22:26.401802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.401778    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"]
Oct 30 21:22:26.404588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.404546    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:26.404684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.404635    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:26.404684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.404677    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:26.406373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.406345    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.408764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.408708    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"]
Oct 30 21:22:26.417019 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:26.416991    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa530f5f_e62a_4321_830d_19f4b987dad8.slice/crio-12a43d8112e33337747617026e1ee6e395dda59d64d202ba1a7e11f1bfd4b5fd WatchSource:0}: Error finding container 12a43d8112e33337747617026e1ee6e395dda59d64d202ba1a7e11f1bfd4b5fd: Status 404 returned error can't find the container with id 12a43d8112e33337747617026e1ee6e395dda59d64d202ba1a7e11f1bfd4b5fd
Oct 30 21:22:26.419685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.419656    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:26.419769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.419715    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:26.419769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.419745    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:26.560996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.560945    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"]
Oct 30 21:22:26.573373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.569121    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"]
Oct 30 21:22:26.591542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:26.591103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.631028 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:26.630994    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:22:27.155025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.154983    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" event={"ID":"5565c23c-7d17-4aa2-9205-099b71e529d8","Type":"ContainerStarted","Data":"b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1"}
Oct 30 21:22:27.155218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.155055    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" event={"ID":"5565c23c-7d17-4aa2-9205-099b71e529d8","Type":"ContainerStarted","Data":"82c33987d65696027623fa8c61c54ebfd0bd71ca819b3ccaadedeb2c452bd246"}
Oct 30 21:22:27.156119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.156097    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:27.158116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.158091    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" event={"ID":"aa530f5f-e62a-4321-830d-19f4b987dad8","Type":"ContainerStarted","Data":"7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49"}
Oct 30 21:22:27.158241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.158136    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" event={"ID":"aa530f5f-e62a-4321-830d-19f4b987dad8","Type":"ContainerStarted","Data":"12a43d8112e33337747617026e1ee6e395dda59d64d202ba1a7e11f1bfd4b5fd"}
Oct 30 21:22:27.161110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.161077    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" event={"ID":"6ddf7906-05f9-40d5-a863-28b79fedcf3f","Type":"ContainerStarted","Data":"4e9e4a6722b64b1721016cb3fcbb7394a11defe1f99199029706326c2b8331a6"}
Oct 30 21:22:27.161227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.161128    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" event={"ID":"6ddf7906-05f9-40d5-a863-28b79fedcf3f","Type":"ContainerStarted","Data":"adaba52dbd367ecc257d7b932a4d2ce42d33c0dea7e3748e7a21d69fb0027b25"}
Oct 30 21:22:27.165073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.165034    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" event={"ID":"84ae85b9-870e-42b1-b170-7e72d4927525","Type":"ContainerStarted","Data":"cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416"}
Oct 30 21:22:27.165196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.165081    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" event={"ID":"84ae85b9-870e-42b1-b170-7e72d4927525","Type":"ContainerStarted","Data":"e270faafcbf714df6e9880255cb3f3e6261965229f7c99559dea4a9feb119dcc"}
Oct 30 21:22:27.165196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.165142    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:27.255126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.255073    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" podStartSLOduration=2.255056202 podStartE2EDuration="2.255056202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:27.222853658 +0000 UTC m=+1329.997253600" watchObservedRunningTime="2026-10-30 21:22:27.255056202 +0000 UTC m=+1330.029456141"
Oct 30 21:22:27.272971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.269518    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podStartSLOduration=2.26949773 podStartE2EDuration="2.26949773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:27.264693987 +0000 UTC m=+1330.039093926" watchObservedRunningTime="2026-10-30 21:22:27.26949773 +0000 UTC m=+1330.043897671"
Oct 30 21:22:27.363703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.363640    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podStartSLOduration=2.363618901 podStartE2EDuration="2.363618901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:27.341843932 +0000 UTC m=+1330.116243866" watchObservedRunningTime="2026-10-30 21:22:27.363618901 +0000 UTC m=+1330.138018840"
Oct 30 21:22:27.713852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.712518    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:27.713852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.712637    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:27.756868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.747939    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:22:27.756868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.748024    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 21:22:27.938929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.938894    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:27.939737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.939717    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:27.945714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:27.943342    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" podStartSLOduration=2.943324443 podStartE2EDuration="2.943324443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:27.398101464 +0000 UTC m=+1330.172501439" watchObservedRunningTime="2026-10-30 21:22:27.943324443 +0000 UTC m=+1330.717724382"
Oct 30 21:22:28.345990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:28.345948    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:28.553415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:28.553380    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:22:28.553728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:28.553676    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 21:22:28.592593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:28.592558    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:28.592959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:28.592931    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:29.551135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:29.551101    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:22:29.551539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:29.551161    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 21:22:29.613854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:29.611971    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:29.613854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:29.612057    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:30.129170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:30.129127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:30.553055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:30.551716    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:22:30.553055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:30.551776    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 21:22:30.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:30.595608    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:30.615951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:30.612247    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:31.551608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:31.551568    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:22:31.551774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:31.551626    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 21:22:31.613858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:31.598278    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:31.613858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:31.598360    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:32.119855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:32.108846    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:32.550906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:32.542545    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:32.551797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:32.551760    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 21:22:32.551944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:32.551822    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 21:22:32.626785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:32.626743    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:32.627317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:32.626820    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:33.086861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.084464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:33.281881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.276940    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"]
Oct 30 21:22:33.533443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.523186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:33.533443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.525205    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"]
Oct 30 21:22:33.538585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.538548    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"]
Oct 30 21:22:33.562225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.562198    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"]
Oct 30 21:22:33.562605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.562524    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.589962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.589932    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ca18b3-openshift-default-dockercfg-kl4ft\""
Oct 30 21:22:33.591805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.591776    4549 patch_prober.go:29] interesting pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:22:33.592029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.592000    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:22:33.635551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635515    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635601    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635692    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635784    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635816    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635877    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635909    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.635961    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.636052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.636001    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxlj7\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.666367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.666332    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:33.667388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.667342    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:33.669322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.669301    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:22:33.738161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738125    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738214    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738304    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738423    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738452    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738485    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738514    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738552    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.738605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.738579    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxlj7\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.739384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.739361    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.739498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.739423    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.739564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.739509    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.739564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.739544    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.739645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.739618    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.741909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.741881    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.742024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.742001    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.774858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.767439    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxlj7\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.774858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.773774    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:33.819902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.815670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:33.906856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:33.905257    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:34.491519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.491423    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:34.657259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.657228    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"]
Oct 30 21:22:34.658092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.658055    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:34.660052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.660034    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:34.662077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.662059    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:22:34.669460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.669417    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:34.669564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.669514    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:34.669564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.669557    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:22:34.678096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:34.678066    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" event={"ID":"893d4eeb-8daf-4865-b633-de3eed97661e","Type":"ContainerStarted","Data":"22e0ff1fbebb78be1052bbbbabb416953e5595e0320fb05bd3f2e39e884a2fb4"}
Oct 30 21:22:35.689530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:35.689482    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" event={"ID":"893d4eeb-8daf-4865-b633-de3eed97661e","Type":"ContainerStarted","Data":"c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"}
Oct 30 21:22:35.933913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:35.933877    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:35.959189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:35.959100    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:36.018343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:36.016407    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podStartSLOduration=3.016390439 podStartE2EDuration="3.016390439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:35.756399107 +0000 UTC m=+1338.530799047" watchObservedRunningTime="2026-10-30 21:22:36.016390439 +0000 UTC m=+1338.790790378"
Oct 30 21:22:36.160503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:36.155488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:36.697690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:36.697645    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:36.698580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:36.698560    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:22:37.668581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:37.668543    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.196067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:38.196032    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:22:38.205462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:38.205398    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.278069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:38.278025    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerName="payload-processing" containerID="cri-o://7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff" gracePeriod=25
Oct 30 21:22:38.410247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:38.410210    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.531749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:38.531640    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerName="payload-pre-processing" containerID="cri-o://816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b" gracePeriod=25
Oct 30 21:22:39.227115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:39.227087    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:22:39.505306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:39.505215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:39.823789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:39.823733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:39.973234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:39.973134    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:41.536072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:41.536040    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:43.021257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:43.021229    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:44.206755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:44.206723    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:22:44.207165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:44.206720    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:22:44.452690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:44.452662    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:46.014841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:46.014795    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:47.525899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:47.525856    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:47.672857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:47.670025    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.309082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:49.309033    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.577557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:49.577516    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.767252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:49.767199    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:51.136357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:51.136323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:52.915200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:52.915165    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:53.121723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:53.121692    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:54.219395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:54.219339    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:22:54.220516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:54.220481    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:22:54.717967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:54.717925    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:56.659567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.656967    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"]
Oct 30 21:22:56.668751 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:56.668675    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:22:56.693905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.670151    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"]
Oct 30 21:22:56.693905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.673971    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.712857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.709231    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-75c17d-serving-cert\""
Oct 30 21:22:56.771449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.771390    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.771769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.771743    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.772153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.772126    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.772410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.772372    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmpgx\" (UniqueName: \"kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.777588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.777549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:56.890366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.890323    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmpgx\" (UniqueName: \"kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.890787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.890766    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.890969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.890952    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.891208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.891192    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.892527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:56.892501    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:56.940922 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:56.892756    4549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-serving-cert: secret "maas-api-e2e-shared-b-w2-75c17d-serving-cert" not found
Oct 30 21:22:57.061470 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:57.061441    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls podName:deeb9b01-ff43-4ec4-ab26-eaecb69755eb nodeName:}" failed. No retries permitted until 2026-10-30 21:22:57.561419226 +0000 UTC m=+1360.335819158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls") pod "maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" (UID: "deeb9b01-ff43-4ec4-ab26-eaecb69755eb") : secret "maas-api-e2e-shared-b-w2-75c17d-serving-cert" not found
Oct 30 21:22:57.061470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.061322    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:57.061751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.055566    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmpgx\" (UniqueName: \"kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:57.069426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.069385    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:57.201676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.201568    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:57.645102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.645060    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:57.647706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.647678    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:57.938494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:57.938387    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:58.477259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.477220    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"]
Oct 30 21:22:58.481768 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:22:58.481736    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeeb9b01_ff43_4ec4_ab26_eaecb69755eb.slice/crio-f71dfbaf354239dfaaa228c9fad5398b2484b1c6a8e08bd3f876f6104fe56a28 WatchSource:0}: Error finding container f71dfbaf354239dfaaa228c9fad5398b2484b1c6a8e08bd3f876f6104fe56a28: Status 404 returned error can't find the container with id f71dfbaf354239dfaaa228c9fad5398b2484b1c6a8e08bd3f876f6104fe56a28
Oct 30 21:22:58.645820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.645795    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:22:58.771001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.770959    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume\") pod \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") "
Oct 30 21:22:58.771307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.771285    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6\" (UniqueName: \"kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6\") pod \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\" (UID: \"f662f5ff-5084-4c6a-8749-c1a79feadf1a\") "
Oct 30 21:22:58.771429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.771374    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume" pod "f662f5ff-5084-4c6a-8749-c1a79feadf1a" (UID: "f662f5ff-5084-4c6a-8749-c1a79feadf1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:58.772064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.772037    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f662f5ff-5084-4c6a-8749-c1a79feadf1a-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:58.774015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.773984    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6" pod "f662f5ff-5084-4c6a-8749-c1a79feadf1a" (UID: "f662f5ff-5084-4c6a-8749-c1a79feadf1a"). InnerVolumeSpecName "kube-api-access-d8nw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:58.813803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.813776    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:22:58.850235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.850207    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.880548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.877427    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z\" (UniqueName: \"kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z\") pod \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") "
Oct 30 21:22:58.918876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.894578    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume\") pod \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\" (UID: \"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac\") "
Oct 30 21:22:58.918876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.896124    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8nw6\" (UniqueName: \"kubernetes.io/projected/f662f5ff-5084-4c6a-8749-c1a79feadf1a-kube-api-access-d8nw6\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:58.918876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.916140    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume" pod "e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" (UID: "e0d1cd9e-5204-4b0d-a225-c897fc0f98ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:58.952935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:58.943292    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z" pod "e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" (UID: "e0d1cd9e-5204-4b0d-a225-c897fc0f98ac"). InnerVolumeSpecName "kube-api-access-6dp9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:59.000640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.000564    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:59.000640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.000610    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dp9z\" (UniqueName: \"kubernetes.io/projected/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac-kube-api-access-6dp9z\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:22:59.237881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.237751    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" event={"ID":"deeb9b01-ff43-4ec4-ab26-eaecb69755eb","Type":"ContainerStarted","Data":"913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf"}
Oct 30 21:22:59.237881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.237816    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" event={"ID":"deeb9b01-ff43-4ec4-ab26-eaecb69755eb","Type":"ContainerStarted","Data":"f71dfbaf354239dfaaa228c9fad5398b2484b1c6a8e08bd3f876f6104fe56a28"}
Oct 30 21:22:59.237881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.237860    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:22:59.239354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.239274    4549 generic.go:356] "Generic (PLEG): container finished" podID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerID="7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff" exitCode=0
Oct 30 21:22:59.239354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.239311    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"
Oct 30 21:22:59.239354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.239351    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" event={"ID":"f662f5ff-5084-4c6a-8749-c1a79feadf1a","Type":"ContainerDied","Data":"7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff"}
Oct 30 21:22:59.239603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.239405    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg" event={"ID":"f662f5ff-5084-4c6a-8749-c1a79feadf1a","Type":"ContainerDied","Data":"386aa885f6828a4715bde8d713eab99c94c3f218da39a72e4c34b30bff37c844"}
Oct 30 21:22:59.239603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.239436    4549 scope.go:122] "RemoveContainer" containerID="7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff"
Oct 30 21:22:59.241195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.241166    4549 generic.go:356] "Generic (PLEG): container finished" podID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerID="816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b" exitCode=0
Oct 30 21:22:59.241325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.241220    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"
Oct 30 21:22:59.241325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.241241    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" event={"ID":"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac","Type":"ContainerDied","Data":"816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b"}
Oct 30 21:22:59.241325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.241286    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl" event={"ID":"e0d1cd9e-5204-4b0d-a225-c897fc0f98ac","Type":"ContainerDied","Data":"a1f63e6087ce9e798937fdc2757ee07e3b679f8f00f701a5ac28e3265dc79772"}
Oct 30 21:22:59.252431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.252409    4549 scope.go:122] "RemoveContainer" containerID="7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff"
Oct 30 21:22:59.252804 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:59.252772    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff\": container with ID starting with 7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff not found: ID does not exist" containerID="7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff"
Oct 30 21:22:59.252916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.252814    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff"} err="failed to get container status \"7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff\": rpc error: code = NotFound desc = could not find container \"7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff\": container with ID starting with 7575a11dd68f5721138e909853caae03f3a5f9b6a230c5b32861cd94578c1fff not found: ID does not exist"
Oct 30 21:22:59.252916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.252884    4549 scope.go:122] "RemoveContainer" containerID="816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b"
Oct 30 21:22:59.263205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.263181    4549 scope.go:122] "RemoveContainer" containerID="816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b"
Oct 30 21:22:59.263496 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:22:59.263478    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b\": container with ID starting with 816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b not found: ID does not exist" containerID="816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b"
Oct 30 21:22:59.263541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.263505    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b"} err="failed to get container status \"816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b\": rpc error: code = NotFound desc = could not find container \"816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b\": container with ID starting with 816438319f53816c21de8f5a204bd9a2e4f4649f0f9d143e712719435538d90b not found: ID does not exist"
Oct 30 21:22:59.281650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.281585    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" podStartSLOduration=2.9410938 podStartE2EDuration="3.281568513s" totalImagesPullingTime="340.474713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:59.277330548 +0000 UTC m=+1362.051730489" watchObservedRunningTime="2026-10-30 21:22:59.281568513 +0000 UTC m=+1362.055968499"
Oct 30 21:22:59.317104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.317072    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"]
Oct 30 21:22:59.321651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.321619    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg"]
Oct 30 21:22:59.351178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.351139    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" path="/var/lib/kubelet/pods/f662f5ff-5084-4c6a-8749-c1a79feadf1a/volumes"
Oct 30 21:22:59.351630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.351607    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"]
Oct 30 21:22:59.356345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:22:59.356311    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl"]
Oct 30 21:23:00.535244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:00.535210    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:00.754431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:00.754402    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"]
Oct 30 21:23:00.754635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:00.754611    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy" containerID="cri-o://be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029" gracePeriod=30
Oct 30 21:23:01.354702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:01.354659    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" path="/var/lib/kubelet/pods/e0d1cd9e-5204-4b0d-a225-c897fc0f98ac/volumes"
Oct 30 21:23:02.117589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:02.117546    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:02.311918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:02.311816    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"]
Oct 30 21:23:02.312464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:02.312421    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy" containerID="cri-o://b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab" gracePeriod=30
Oct 30 21:23:02.322860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:02.319127    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"]
Oct 30 21:23:02.322860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:02.320171    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy" containerID="cri-o://9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9" gracePeriod=30
Oct 30 21:23:03.630363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:03.630327    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:05.053682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:05.053643    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:05.258711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:05.258677    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:23:06.069969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.069942    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:23:06.074152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074128    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074193    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074292    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074343    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074368    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074427    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074472    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074489    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:06.074672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074581    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074604    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:06.074672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074652    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert\") pod \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\" (UID: \"7b8f2f70-bf79-4576-ae89-9f43a1713e5c\") "
Oct 30 21:23:06.074823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074757    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:06.074936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.074913    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:06.075311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.075225    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:06.075395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.075367    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.075395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.075388    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.075477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.075403    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.075477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.075416    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.076736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.076703    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:23:06.076736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.076728    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "kube-api-access-jshmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:06.077089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.077059    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:06.077089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.077084    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token" pod "7b8f2f70-bf79-4576-ae89-9f43a1713e5c" (UID: "7b8f2f70-bf79-4576-ae89-9f43a1713e5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:06.177110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.177074    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.177110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.177107    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.177110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.177116    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.177357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.177125    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.177357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.177134    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jshmz\" (UniqueName: \"kubernetes.io/projected/7b8f2f70-bf79-4576-ae89-9f43a1713e5c-kube-api-access-jshmz\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:06.368875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.368816    4549 generic.go:356] "Generic (PLEG): container finished" podID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerID="be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029" exitCode=0
Oct 30 21:23:06.368875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.368868    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" event={"ID":"7b8f2f70-bf79-4576-ae89-9f43a1713e5c","Type":"ContainerDied","Data":"be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029"}
Oct 30 21:23:06.369166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.368917    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"
Oct 30 21:23:06.369166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.368929    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w" event={"ID":"7b8f2f70-bf79-4576-ae89-9f43a1713e5c","Type":"ContainerDied","Data":"2f03ecf27d543e26840304ad81a843d76a4a8eaba4717b812c744e8179919f9d"}
Oct 30 21:23:06.369166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.368957    4549 scope.go:122] "RemoveContainer" containerID="be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029"
Oct 30 21:23:06.381396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.381373    4549 scope.go:122] "RemoveContainer" containerID="be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029"
Oct 30 21:23:06.381821 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:23:06.381799    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029\": container with ID starting with be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029 not found: ID does not exist" containerID="be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029"
Oct 30 21:23:06.381821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.381848    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029"} err="failed to get container status \"be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029\": rpc error: code = NotFound desc = could not find container \"be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029\": container with ID starting with be4b26a549c4ddb3bb0859ee508bd36753c7ea0786f07361fc3cb1f71b55f029 not found: ID does not exist"
Oct 30 21:23:06.424751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.424716    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"]
Oct 30 21:23:06.436599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.436565    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w"]
Oct 30 21:23:06.614526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.614456    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:06.746294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:06.746184    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:07.349152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.349119    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" path="/var/lib/kubelet/pods/7b8f2f70-bf79-4576-ae89-9f43a1713e5c/volumes"
Oct 30 21:23:07.661424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.661390    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:23:07.679471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.679449    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:23:07.701467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701433    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701493    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701519    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701635    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701718    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701771    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701794    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701787    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701871    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701915    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701957    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701989    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf\") pod \"13226cae-6ab7-44ac-9070-926073140741\" (UID: \"13226cae-6ab7-44ac-9070-926073140741\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702030    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702080    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.701952    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702117    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702157    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702161    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702189    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:07.702263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702232    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702348    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702433    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702470    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702575    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:07.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.702633    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo\") pod \"234f9898-40ad-4244-879d-ddc6cdb27830\" (UID: \"234f9898-40ad-4244-879d-ddc6cdb27830\") "
Oct 30 21:23:07.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703010    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703526    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13226cae-6ab7-44ac-9070-926073140741-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703549    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703562    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703574    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703588    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234f9898-40ad-4244-879d-ddc6cdb27830-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703600    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703612    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.703671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.703625    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.706005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.704734    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.706005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.705252    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "kube-api-access-5tqsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:07.706005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.705253    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.706245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.706043    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:23:07.706245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.706112    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:23:07.706359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.706302    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.708342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.708304    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token" pod "13226cae-6ab7-44ac-9070-926073140741" (UID: "13226cae-6ab7-44ac-9070-926073140741"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:07.708666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.708552    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "kube-api-access-rhf48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:07.711355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.709658    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:23:07.711355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.710643    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token" pod "234f9898-40ad-4244-879d-ddc6cdb27830" (UID: "234f9898-40ad-4244-879d-ddc6cdb27830"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:07.805982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805903    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13226cae-6ab7-44ac-9070-926073140741-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.805982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805932    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13226cae-6ab7-44ac-9070-926073140741-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.805982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805945    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.805982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805957    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tqsf\" (UniqueName: \"kubernetes.io/projected/13226cae-6ab7-44ac-9070-926073140741-kube-api-access-5tqsf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.805982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805972    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.805982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805984    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.806509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.805997    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234f9898-40ad-4244-879d-ddc6cdb27830-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.806509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.806009    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.806509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.806021    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhf48\" (UniqueName: \"kubernetes.io/projected/234f9898-40ad-4244-879d-ddc6cdb27830-kube-api-access-rhf48\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:07.806509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:07.806033    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234f9898-40ad-4244-879d-ddc6cdb27830-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:23:08.038484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.038404    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:08.386559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.386525    4549 generic.go:356] "Generic (PLEG): container finished" podID="13226cae-6ab7-44ac-9070-926073140741" containerID="b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab" exitCode=0
Oct 30 21:23:08.387091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.386574    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"
Oct 30 21:23:08.387091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.386612    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" event={"ID":"13226cae-6ab7-44ac-9070-926073140741","Type":"ContainerDied","Data":"b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab"}
Oct 30 21:23:08.387091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.386683    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m" event={"ID":"13226cae-6ab7-44ac-9070-926073140741","Type":"ContainerDied","Data":"0bc5ec03bb1e26f3ecda2147ee2fd6289137f4aea52226e3972a264b88017705"}
Oct 30 21:23:08.387091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.386718    4549 scope.go:122] "RemoveContainer" containerID="b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab"
Oct 30 21:23:08.389732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.389708    4549 generic.go:356] "Generic (PLEG): container finished" podID="234f9898-40ad-4244-879d-ddc6cdb27830" containerID="9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9" exitCode=0
Oct 30 21:23:08.390282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.389915    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" event={"ID":"234f9898-40ad-4244-879d-ddc6cdb27830","Type":"ContainerDied","Data":"9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9"}
Oct 30 21:23:08.390461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.390443    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2" event={"ID":"234f9898-40ad-4244-879d-ddc6cdb27830","Type":"ContainerDied","Data":"8d1b882e215c292bbf62c7ae7972db4c4019e8d4be0a7096c5b259feeb69d7fd"}
Oct 30 21:23:08.390585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.390002    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"
Oct 30 21:23:08.423782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.419083    4549 scope.go:122] "RemoveContainer" containerID="b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab"
Oct 30 21:23:08.423782 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:23:08.422069    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab\": container with ID starting with b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab not found: ID does not exist" containerID="b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab"
Oct 30 21:23:08.423782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.422108    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab"} err="failed to get container status \"b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab\": rpc error: code = NotFound desc = could not find container \"b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab\": container with ID starting with b5ba8df1a96525d0a362bd951d27ad5229cde728ba8036ccad26f8ec75c562ab not found: ID does not exist"
Oct 30 21:23:08.423782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.422133    4549 scope.go:122] "RemoveContainer" containerID="9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9"
Oct 30 21:23:08.435156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.435132    4549 scope.go:122] "RemoveContainer" containerID="9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9"
Oct 30 21:23:08.435445 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:23:08.435425    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9\": container with ID starting with 9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9 not found: ID does not exist" containerID="9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9"
Oct 30 21:23:08.435505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.435454    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9"} err="failed to get container status \"9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9\": rpc error: code = NotFound desc = could not find container \"9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9\": container with ID starting with 9324fbb425c022dcc022a60116028e4ff5cdabef1319c2be3e9a93b5d29a33b9 not found: ID does not exist"
Oct 30 21:23:08.471527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.471499    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"]
Oct 30 21:23:08.483520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.483499    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m"]
Oct 30 21:23:08.510913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.510883    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"]
Oct 30 21:23:08.515304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:08.515279    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2"]
Oct 30 21:23:09.350606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.350566    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13226cae-6ab7-44ac-9070-926073140741" path="/var/lib/kubelet/pods/13226cae-6ab7-44ac-9070-926073140741/volumes"
Oct 30 21:23:09.351152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.351128    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" path="/var/lib/kubelet/pods/234f9898-40ad-4244-879d-ddc6cdb27830/volumes"
Oct 30 21:23:09.480402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.480367    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"]
Oct 30 21:23:09.481127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481111    4549 cpu_manager.go:414] "containerMap: removing container" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy"
Oct 30 21:23:09.481127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481127    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481136    4549 cpu_manager.go:414] "containerMap: removing container" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481141    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481161    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerName="payload-pre-processing"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481166    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerName="payload-pre-processing"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481175    4549 cpu_manager.go:414] "containerMap: removing container" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerName="payload-processing"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481180    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerName="payload-processing"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481195    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy"
Oct 30 21:23:09.481244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481200    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy"
Oct 30 21:23:09.481730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481290    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b8f2f70-bf79-4576-ae89-9f43a1713e5c" containerName="istio-proxy"
Oct 30 21:23:09.481730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481305    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="234f9898-40ad-4244-879d-ddc6cdb27830" containerName="istio-proxy"
Oct 30 21:23:09.481730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481311    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="13226cae-6ab7-44ac-9070-926073140741" containerName="istio-proxy"
Oct 30 21:23:09.481730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481319    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0d1cd9e-5204-4b0d-a225-c897fc0f98ac" containerName="payload-pre-processing"
Oct 30 21:23:09.481730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.481328    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="f662f5ff-5084-4c6a-8749-c1a79feadf1a" containerName="payload-processing"
Oct 30 21:23:09.486790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.486772    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.503950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.503625    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ca18b3-serving-cert\""
Oct 30 21:23:09.520713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.520687    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"]
Oct 30 21:23:09.545510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.543205    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.545510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.543307    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.545510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.543434    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtgld\" (UniqueName: \"kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.545510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.543493    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.584591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.584561    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.645078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.645015    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.645078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.645073    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.645228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.645190    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgld\" (UniqueName: \"kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.645262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.645240    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.645938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.645911    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.647743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.647721    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.647870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.647720    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.664295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.664276    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtgld\" (UniqueName: \"kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld\") pod \"maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.765625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.765589    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.803774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.803735    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:09.917592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:09.917487    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.179764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:10.179734    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"]
Oct 30 21:23:10.179973 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:23:10.179951    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c07aa3a_41bf_416e_999f_0e7e7a08fa37.slice/crio-1530054779d2e07c2006948b249ff389628de6f9af4aedec939fe30268965b52 WatchSource:0}: Error finding container 1530054779d2e07c2006948b249ff389628de6f9af4aedec939fe30268965b52: Status 404 returned error can't find the container with id 1530054779d2e07c2006948b249ff389628de6f9af4aedec939fe30268965b52
Oct 30 21:23:10.409909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:10.409869    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv" event={"ID":"2c07aa3a-41bf-416e-999f-0e7e7a08fa37","Type":"ContainerStarted","Data":"1530054779d2e07c2006948b249ff389628de6f9af4aedec939fe30268965b52"}
Oct 30 21:23:11.283056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:11.283013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:11.428154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:11.428113    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv" event={"ID":"2c07aa3a-41bf-416e-999f-0e7e7a08fa37","Type":"ContainerStarted","Data":"018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a"}
Oct 30 21:23:11.428341 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:11.428316    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:11.470212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:11.470163    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv" podStartSLOduration=2.126669959 podStartE2EDuration="2.470148025s" totalImagesPullingTime="343.478066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:11.467582477 +0000 UTC m=+1374.241982416" watchObservedRunningTime="2026-10-30 21:23:11.470148025 +0000 UTC m=+1374.244547963"
Oct 30 21:23:12.809083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:12.809046    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.313223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:14.313164    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:15.803792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:15.803760    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.272859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:17.266916    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.442086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:17.442059    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:23:18.820658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:18.820627    4549 scope.go:122] "RemoveContainer" containerID="b9326704c2e6660318ce9f1ef76978e94dcfe4d63ca5e6c1bb9c8a29f00d0018"
Oct 30 21:23:18.821634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:18.821604    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.300506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:20.299578    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.093287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:22.093244    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.838353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:22.838315    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.963855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:23.960351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:25.456150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:25.456118    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:26.582386 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:23:26.582345    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:23:27.221347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:27.221298    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.176248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:28.176204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.697535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:28.697490    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.832852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:28.831705    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.000279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.000162    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.091283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.091249    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"]
Oct 30 21:23:29.099107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.099080    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.120755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.120731    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"]
Oct 30 21:23:29.123094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.123072    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-95692f-openshift-default-dockercfg-q7k6z\""
Oct 30 21:23:29.191807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.191769    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.233854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.233802    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234012    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234118    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234184    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234217    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234251    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234329    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234364    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbmpp\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.234414 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.234382    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341098    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341203    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341252    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341294    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341343    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341441    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341511    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbmpp\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341538    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.341750    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.342284    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.342856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.342519    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.344173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.344148    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.344722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.344702    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.355901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.351711    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.357270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.357245    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.365859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.357626    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.394885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.371203    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.451852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.447111    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbmpp\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.471078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.470009    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token\") pod \"e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") " pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:29.719530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:29.719440    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:30.046095 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:23:30.046044    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0080a530_5406_4ecb_8fcc_ad4fa5fb0793.slice/crio-df0a68419ec911f622a463d59e285b27a9021935b4fb1bb1fc8810dde59ecd52 WatchSource:0}: Error finding container df0a68419ec911f622a463d59e285b27a9021935b4fb1bb1fc8810dde59ecd52: Status 404 returned error can't find the container with id df0a68419ec911f622a463d59e285b27a9021935b4fb1bb1fc8810dde59ecd52
Oct 30 21:23:30.051103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.051061    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:30.051315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.051293    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:30.051461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.051443    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:30.074572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.074538    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"]
Oct 30 21:23:30.277840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.277787    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:30.730892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.730852    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" event={"ID":"0080a530-5406-4ecb-8fcc-ad4fa5fb0793","Type":"ContainerStarted","Data":"0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0"}
Oct 30 21:23:30.731208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.731170    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" event={"ID":"0080a530-5406-4ecb-8fcc-ad4fa5fb0793","Type":"ContainerStarted","Data":"df0a68419ec911f622a463d59e285b27a9021935b4fb1bb1fc8810dde59ecd52"}
Oct 30 21:23:30.902861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:30.902754    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podStartSLOduration=1.902728811 podStartE2EDuration="1.902728811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:30.882089418 +0000 UTC m=+1393.656489355" watchObservedRunningTime="2026-10-30 21:23:30.902728811 +0000 UTC m=+1393.677128752"
Oct 30 21:23:31.421227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.421193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:31.548694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.548657    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:31.682189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.682062    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:31.720654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.720627    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:31.727388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.727356    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:31.739198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.739171    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:31.740205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:31.740184    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:23:33.312468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:33.312434    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:33.451633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:33.451598    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:34.813611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:34.813566    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.015177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:35.015139    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.302420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:35.302388    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.528855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:35.527813    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.748410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:35.748371    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.958367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:35.958329    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:36.561860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:36.559885    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:37.076198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.076149    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:23:37.177638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.177590    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:37.194060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.194014    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:37.287854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.264471    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"]
Oct 30 21:23:37.539917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.534961    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:37.620761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.620707    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:23:37.684271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.670887    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.732876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.723013    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:37.753860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.750372    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c31999-openshift-default-dockercfg-j7gl5\""
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.913120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.931902    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.931966    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932161    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932232    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnflj\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932279    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932325    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932418    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932468    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:37.937056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:37.932669    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.038748    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.038873    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnflj\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.038931    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.038979    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.039096    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.039146    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.039350    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.039481    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.039533    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.066899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.044418    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.073764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.073734    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.076260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.076230    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.080730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.080696    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:38.084297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.084269    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.105992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.101029    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:23:38.123949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.123748    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.136856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.130672    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"]
Oct 30 21:23:38.170148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.170062    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:23:38.171683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.158303    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.176950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.153545    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnflj\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.186002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.185959    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:23:38.195867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.187769    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:23:38.224921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.196314    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:38.245306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.198490    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.262079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.240401    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token\") pod \"e2e-inf-a-c31999-openshift-default-967c87455-k4v94\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") " pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.262375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.246604    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:23:38.326360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.326320    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:38.459918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.434126    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:38.460392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.436217    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:38.526167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.451571    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Oct 30 21:23:38.526167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.494758    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:38.558068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.558017    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:38.583864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.559115    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:23:38.744925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.739928    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:38.792724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.791451    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:38.802982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.793047    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:23:38.828875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.828843    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:23:39.003909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:38.961256    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:39.124853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.123847    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:39.343859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.342901    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:39.370872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.344458    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:39.606921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.574900    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:39.726862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.705894    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:39.752858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.729454    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:39.768964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.768917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:39.769477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.769418    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:39.874055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:39.874011    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:40.577705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:40.577076    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:40.792697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:40.768509    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:40.792697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:40.769648    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:40.868486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:40.813723    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:40.928991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:40.895279    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:41.044894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.026370    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:23:41.116469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.107643    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:23:41.117525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.117483    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:23:41.448908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.448811    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:23:41.660281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.660241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:41.867522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.866787    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:41.878568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:41.866917    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.103108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.103059    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-c5dtn" podUID="0f292b66-399c-4566-8921-3f04abdaed5a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded"
Oct 30 21:23:42.151094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.150991    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.153023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.152980    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:23:42.188283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.188235    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:23:42.356420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.356372    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4wgct" podUID="569e7b40-df79-438b-a350-7f4cf8202c9c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.457915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.449918    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:23:42.462031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.458216    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.484318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.484272    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.541855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.534596    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:42.651908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.648984    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:42.651908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.649119    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:42.652227 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:23:42.597136    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="3.101s"
Oct 30 21:23:42.652498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.652483    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:23:42.652668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.652654    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:23:42.652937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.652379    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.884178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.884127    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.885304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:42.885270    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.225137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.151820    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.225137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.155934    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.225137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.201145    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.251422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.230334    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" event={"ID":"2be14cc1-2729-4b2a-a37f-b01dd0b27a84","Type":"ContainerStarted","Data":"84e608f651b1142c8ad64ff80e34c984108810d3704e332119351dc716e1d245"}
Oct 30 21:23:43.251422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.230427    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"]
Oct 30 21:23:43.375868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.357327    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.526344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.517932    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.627901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.602888    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.886932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.886880    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.897066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.897026    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:23:43.897270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:43.897107    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:23:44.020310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.015485    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/health\": context deadline exceeded"
Oct 30 21:23:44.158223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.158120    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.158946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.158431    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:23:44.218875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.208641    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.540097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.518895    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.650859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.628675    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:23:44.903052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.891434    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.919988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.919933    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:44.931008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:44.930962    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:45.073065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.073023    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" event={"ID":"2be14cc1-2729-4b2a-a37f-b01dd0b27a84","Type":"ContainerStarted","Data":"e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"}
Oct 30 21:23:45.146438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.145922    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podStartSLOduration=9.145899993 podStartE2EDuration="9.145899993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:45.129209216 +0000 UTC m=+1407.903609156" watchObservedRunningTime="2026-10-30 21:23:45.145899993 +0000 UTC m=+1407.920299936"
Oct 30 21:23:45.163997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.163943    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:23:45.166581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.166523    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:23:45.237289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.235959    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:23:45.582988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.582939    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:45.631481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.631433    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:45.661860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.658451    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:45.661860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.658581    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:45.661860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.660763    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:23:45.686178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.686150    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:23:45.745855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.738633    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:23:45.857961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.857874    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:45.872128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.868199    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:23:45.897497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.895126    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:45.942562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.942527    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:23:45.943979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:45.943885    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:23:46.088044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:46.087727    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:46.088905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:46.088882    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:23:47.742347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:47.742282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:47.940858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:47.920102    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:48.806576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.806538    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"]
Oct 30 21:23:48.838540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.838508    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"]
Oct 30 21:23:48.838540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.838565    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.954883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.954814    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.954906    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.954966    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.954990    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.955047    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnt96\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.955197    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.955250    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.955459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.955404    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:48.957315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:48.955891    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058436    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058518    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058671    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058787    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058889    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058935    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.058987    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.059010    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.059445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.059063    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnt96\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.060185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.059692    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.061234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.060318    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.061234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.060673    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.061234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.060769    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.061234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.060793    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.067092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.067051    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.068141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.068113    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.081305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.081267    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnt96\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.082145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.082116    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.184183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.184144    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:49.452078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.451477    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.679487 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.679357    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.835509 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:23:49.835454    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bf5ba2_23d2_4d01_a9eb_31a8c4c71be7.slice/crio-35a5e5155ded2c25d85cda5752c9d0f75bd7c216c660acaf582d9a4086d0d5eb WatchSource:0}: Error finding container 35a5e5155ded2c25d85cda5752c9d0f75bd7c216c660acaf582d9a4086d0d5eb: Status 404 returned error can't find the container with id 35a5e5155ded2c25d85cda5752c9d0f75bd7c216c660acaf582d9a4086d0d5eb
Oct 30 21:23:49.878877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.865722    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"]
Oct 30 21:23:49.951045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.944358    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:49.951045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.944471    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:49.951045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:49.944516    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:50.187357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:50.187317    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" event={"ID":"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7","Type":"ContainerStarted","Data":"35a5e5155ded2c25d85cda5752c9d0f75bd7c216c660acaf582d9a4086d0d5eb"}
Oct 30 21:23:50.323851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:50.323727    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:51.226817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:51.220013    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:23:51.226817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:51.220084    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:51.235518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:51.235470    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" event={"ID":"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7","Type":"ContainerStarted","Data":"fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1"}
Oct 30 21:23:51.325788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:51.325726    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podStartSLOduration=3.325706033 podStartE2EDuration="3.325706033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:51.295366081 +0000 UTC m=+1414.069766057" watchObservedRunningTime="2026-10-30 21:23:51.325706033 +0000 UTC m=+1414.100105974"
Oct 30 21:23:52.192855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.191946    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:23:52.192855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.192115    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:52.192855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.192251    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:52.244874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.240944    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:23:52.244874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.241016    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:52.914471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.650167    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:52.914471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.764925    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:23:52.914471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:52.764986    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:53.200301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:53.191250    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:53.200301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:53.191308    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:54.188911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.187288    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:54.188911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.187353    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:54.324140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.324103    4549 generic.go:356] "Generic (PLEG): container finished" podID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerID="0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0" exitCode=0
Oct 30 21:23:54.324326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.324181    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" event={"ID":"dd146ff3-1f30-4048-9d66-f1eb69c45132","Type":"ContainerDied","Data":"0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"}
Oct 30 21:23:54.324720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.324702    4549 scope.go:122] "RemoveContainer" containerID="0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"
Oct 30 21:23:54.326260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.326239    4549 generic.go:356] "Generic (PLEG): container finished" podID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerID="922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070" exitCode=0
Oct 30 21:23:54.326380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.326354    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" event={"ID":"f06ed5e1-a07c-43e7-8387-db7a68cc97a7","Type":"ContainerDied","Data":"922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"}
Oct 30 21:23:54.326530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.326497    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.326612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.326581    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.326668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.326627    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.326756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.326738    4549 scope.go:122] "RemoveContainer" containerID="922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"
Oct 30 21:23:54.328629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.328605    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.328745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.328678    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.328745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.328730    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.329125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.329099    4549 generic.go:356] "Generic (PLEG): container finished" podID="893d4eeb-8daf-4865-b633-de3eed97661e" containerID="c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1" exitCode=0
Oct 30 21:23:54.329235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.329178    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" event={"ID":"893d4eeb-8daf-4865-b633-de3eed97661e","Type":"ContainerDied","Data":"c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"}
Oct 30 21:23:54.329390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.329364    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:54.329674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.329657    4549 scope.go:122] "RemoveContainer" containerID="c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"
Oct 30 21:23:54.331468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.331449    4549 generic.go:356] "Generic (PLEG): container finished" podID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerID="1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9" exitCode=0
Oct 30 21:23:54.331702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.331483    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" event={"ID":"eba83462-ebf4-472f-8e5c-cb7b42c25321","Type":"ContainerDied","Data":"1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"}
Oct 30 21:23:54.331871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.331818    4549 scope.go:122] "RemoveContainer" containerID="1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"
Oct 30 21:23:54.331871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.331851    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.332062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.331927    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.332062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.331976    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.342709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.342671    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.342893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.342765    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.342893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.342812    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:23:54.900867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:54.895770    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:55.191387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.191352    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:55.191676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.191647    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:55.351134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.351041    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" event={"ID":"dd146ff3-1f30-4048-9d66-f1eb69c45132","Type":"ContainerStarted","Data":"fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf"}
Oct 30 21:23:55.352802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.352772    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" event={"ID":"f06ed5e1-a07c-43e7-8387-db7a68cc97a7","Type":"ContainerStarted","Data":"f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419"}
Oct 30 21:23:55.356609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.356584    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" event={"ID":"893d4eeb-8daf-4865-b633-de3eed97661e","Type":"ContainerStarted","Data":"81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33"}
Oct 30 21:23:55.360265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.360241    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" event={"ID":"eba83462-ebf4-472f-8e5c-cb7b42c25321","Type":"ContainerStarted","Data":"964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9"}
Oct 30 21:23:55.388088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.388046    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:23:55.388855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.388804    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:23:55.389367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.389035    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:23:55.436293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.436254    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:23:55.436961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.436687    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:23:55.437110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.437006    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:23:55.549442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.549351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:55.914969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.914929    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"]
Oct 30 21:23:55.919746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.919646    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:55.927171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.927139    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:23:55.927988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.927958    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:23:55.928104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.928014    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:23:55.937241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.937213    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-517366-openshift-default-dockercfg-5xb88\""
Oct 30 21:23:55.946370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:55.946335    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"]
Oct 30 21:23:56.014149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.014122    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:56.044102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.044058    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.044413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.044392    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.044642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.044620    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m4zm\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.044889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.044869    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.045096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.045078    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.045333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.045315    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.045532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.045515    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.045771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.045753    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.046107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.046088    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.167853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.167743    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.199968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.185310    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:56.200179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.186037    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.200179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.187390    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:56.200179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.200114    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:56.200426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.200405    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.200584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.200563    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m4zm\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.200801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.200784    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.200995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.200964    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.201306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.201257    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.201548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.201525    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.201797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.201773    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.202157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.202138    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.202952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.202904    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.209131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.203594    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.209247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.209198    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.209971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.209947    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.346178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.346142    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:23:56.416856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.400684    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:56.476625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.449061    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.487009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.452964    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:23:56.487315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.487264    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:23:56.521854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.496624    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:23:56.684901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.684695    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:23:56.686706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.577846    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:23:56.686706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.686632    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:23:56.896806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.896767    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m4zm\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.967036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.897513    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.993136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.897662    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy\") pod \"e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") " pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:56.993400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.930295    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:23:56.993559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:56.993533    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:23:57.154293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.154189    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:23:57.303156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.301148    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:57.477918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.477486    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:57.477918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.477548    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:57.478169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.477974    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:23:57.478169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.478009    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:23:57.483996 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:23:57.483954    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:23:57.484298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.484269    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:57.536430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.536398    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:23:57.536704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.536674    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:23:57.540036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.540006    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:23:57.557960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.557916    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:23:57.720419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.720358    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:23:57.781920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.751120    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:23:57.834706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.834655    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:57.893857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.891314    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:57.926862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.923173    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:57.948256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.948211    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:57.966705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.966660    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:23:57.976074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:57.976025    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:23:58.214859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.206535    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:23:58.214859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.206599    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:23:58.268859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.265192    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:23:58.268859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.265258    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:23:58.318860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.309743    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:23:58.318860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.310080    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:58.318860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.310192    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:23:58.436885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.425446    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:23:58.436885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.425502    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:23:58.467544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.451124    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:23:58.467544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.451178    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:23:58.467544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.455559    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:58.565314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.556950    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:58.565314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.562887    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:23:58.785431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.707818    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": context deadline exceeded"
Oct 30 21:23:58.785431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.733292    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:58.785431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.733766    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:23:58.809856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.798411    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:58.809856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.798859    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:23:58.887183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.866476    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:58.887183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.882849    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:23:58.996438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:58.996398    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:23:59.093852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.093788    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:23:59.101865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.047785    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:59.129859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.116004    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:23:59.334630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.267615    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:23:59.694912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.681902    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:23:59.694912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.641274    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:23:59.842870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.313503    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:59.842870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.776513    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:23:59.842870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.776580    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:00.039919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.963328    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:00.121178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.100242    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:00.121178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.100381    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:00.121178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:23:59.970264    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:24:00.121178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.101280    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:24:00.121178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.054392    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:24:00.121178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.105226    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:00.210249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.210201    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:24:00.227861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.222071    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:24:00.259855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.235367    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:24:00.259855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.235439    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:24:00.259855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.256982    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:24:00.259855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.258514    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:24:00.401855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.384770    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:24:00.433881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.426774    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:24:00.433881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.426868    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:24:00.435387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.435340    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:24:00.489102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.489045    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:00.489495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.489069    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:24:00.489700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.444248    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:24:00.489881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.489860    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:00.691772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.691652    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:00.724903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.717219    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:00.843846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.843775    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:00.949001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:00.947395    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:24:01.153858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.148199    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.356706    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.212020    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.211954    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.314955    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.360629    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.265000    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:01.369651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.326073    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:01.404900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.403156    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:24:01.404900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.403221    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:24:01.460855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.456777    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:24:01.460855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.456867    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:01.528854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.524255    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:01.605920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.586655    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:24:01.605920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.586728    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:24:01.605920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.586751    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"]
Oct 30 21:24:01.684636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.684590    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:01.713894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.701068    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:01.845922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.781536    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:01.870931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:01.849372    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" event={"ID":"7d18cb2f-8a61-494c-9cae-46768da2d8eb","Type":"ContainerStarted","Data":"f1216cdce6e179e2f6eb4a9efbb4b835f93b6c74a1563c545c99fa16910178f4"}
Oct 30 21:24:02.159135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.159032    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:24:02.159716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.159682    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:24:02.274285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.274252    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:24:02.313866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.313566    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:24:02.313866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.313637    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:24:02.313866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.313714    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": context deadline exceeded"
Oct 30 21:24:02.385669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.364405    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:24:02.425375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.425276    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:24:02.425639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.425607    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:24:02.522878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.501410    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:02.522878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.503109    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:24:02.522878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.519708    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:02.544128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.544056    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:02.624186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.503014    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:02.712477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.711075    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:24:02.728396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:02.728349    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:03.181857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.180909    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:24:03.181857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.180971    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:24:03.253539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.253499    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:24:03.254277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.254239    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:24:03.400858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.375542    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:24:03.419319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.393758    4549 patch_prober.go:29] interesting pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 21:24:03.423958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.423909    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 21:24:03.449662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.449472    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:24:03.449662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.449535    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:03.530861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.529065    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:24:03.573959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.556844    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:24:03.635157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.635107    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:24:03.636205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.636179    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:24:03.711602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.711500    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:24:03.730026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.723559    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:24:03.743998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.729620    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:03.941040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.940998    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:24:03.948857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:03.944918    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:24:04.052506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.051131    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:04.260852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.245279    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 21:24:04.260852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.245341    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 21:24:04.334455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.334420    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:24:04.351855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.339323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:04.419186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.419141    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:04.556995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.556939    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:04.714821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.581378    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:04.715844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.602770    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:24:04.716040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.716013    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:04.732077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.732032    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" event={"ID":"7d18cb2f-8a61-494c-9cae-46768da2d8eb","Type":"ContainerStarted","Data":"deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"}
Oct 30 21:24:04.732572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.732549    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:24:04.774763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.765277    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podStartSLOduration=9.765242914 podStartE2EDuration="9.765242914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:04.702118202 +0000 UTC m=+1427.476518140" watchObservedRunningTime="2026-10-30 21:24:04.765242914 +0000 UTC m=+1427.539642853"
Oct 30 21:24:04.817871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.798677    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:24:04.817871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.772214    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:24:04.849858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.772175    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": context deadline exceeded"
Oct 30 21:24:04.850685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.850656    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:24:04.931881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.894204    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:24:04.979520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.954085    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:24:04.979808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.979768    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 21:24:04.980411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:04.956502    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:05.060635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.060244    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:24:05.226702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.224617    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:24:05.226702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.224665    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:24:05.247549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.247466    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:24:05.250945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.250920    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:24:05.262158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.261289    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:24:05.262630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.262304    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:24:05.264857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.263990    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:24:05.266283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.266259    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:24:05.268859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.267034    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:24:05.268859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.267084    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:24:05.436484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.436447    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 21:24:05.436793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.436757    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 21:24:05.464895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.462607    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:24:05.464895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:05.463797    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:24:06.028685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.028640    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:24:06.030872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.030819    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:24:06.036147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.036117    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:24:06.422121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.422080    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:06.440911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.440884    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:24:06.442118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.442096    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:24:06.877538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:06.869243    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.119475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:07.119439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.271391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:07.271298    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.408644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:07.408609    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:08.333315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:08.333280    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:08.916075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:08.915866    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.064270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:09.064236    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.471401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:10.471368    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:12.145326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:12.145262    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.695180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:13.695150    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.866768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:13.866731    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:14.008400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:14.008319    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.425512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:15.425480    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:16.900146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:16.900116    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:17.056592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:17.050362    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.640065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.640026    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.919929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.919852    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"]
Oct 30 21:24:18.924173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.924147    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.946378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.946340    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"]
Oct 30 21:24:18.989603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.989560    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.989760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.989637    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.989760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.989683    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.989760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.989712    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.989937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.989786    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.989937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.989851    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.990245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.990211    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7dbk\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.990395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.990328    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:18.990486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:18.990423    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.046900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.046866    4549 scope.go:122] "RemoveContainer" containerID="0b66ca25b80fb09dc76752fe9b7b205449eea35f46ff7f9e6a3a04dacdea5c37"
Oct 30 21:24:19.092266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092232    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.092266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092284    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.092555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092317    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.092555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092344    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.092555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092411    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.092555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092476    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.092822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092794    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dbk\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092915    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.092940    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.093063    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.093084    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.093288    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.093347    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.093555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.093536    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.094872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.094822    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.095236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.095220    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.112147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.112106    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.112147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.112143    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7dbk\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk\") pod \"e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") " pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.240524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.240442    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:19.585227 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:24:19.585198    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e9a27f_b860_4a10_a145_2513437578ad.slice/crio-22a973273abcc70e54acd6843eac8f2122222b723119aebda78b252f94fcfa24 WatchSource:0}: Error finding container 22a973273abcc70e54acd6843eac8f2122222b723119aebda78b252f94fcfa24: Status 404 returned error can't find the container with id 22a973273abcc70e54acd6843eac8f2122222b723119aebda78b252f94fcfa24
Oct 30 21:24:19.585686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.585661    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"]
Oct 30 21:24:19.588586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.588552    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:19.588686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.588625    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:19.588686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.588657    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:19.717687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.717637    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" event={"ID":"74e9a27f-b860-4a10-a145-2513437578ad","Type":"ContainerStarted","Data":"3da33c59cad0832266c3b6f0302a19ae3be0eeaeb575689048b06caa75bdb2f6"}
Oct 30 21:24:19.718219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.717703    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" event={"ID":"74e9a27f-b860-4a10-a145-2513437578ad","Type":"ContainerStarted","Data":"22a973273abcc70e54acd6843eac8f2122222b723119aebda78b252f94fcfa24"}
Oct 30 21:24:19.763496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:19.762127    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podStartSLOduration=1.762108076 podStartE2EDuration="1.762108076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:19.760290305 +0000 UTC m=+1442.534690245" watchObservedRunningTime="2026-10-30 21:24:19.762108076 +0000 UTC m=+1442.536508017"
Oct 30 21:24:20.240713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:20.240679    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:20.241012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:20.240810    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:20.257853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:20.257047    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:24:20.257853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:20.257108    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:24:21.241454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:21.241414    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:24:21.241854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:21.241473    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:24:21.835125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:21.835089    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:22.240905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:22.240799    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:24:22.240905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:22.240893    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:24:23.241096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:23.241063    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:24:23.241533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:23.241136    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:24:23.292856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:23.292805    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:24.241205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:24.241169    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:24:24.241632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:24.241226    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:24:24.817376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:24.817346    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:25.241596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:25.241514    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:24:25.241596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:25.241578    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:24:26.246898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:26.246865    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:26.247363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:26.247199    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:26.248039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:26.248020    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:24:26.406919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:26.406885    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:26.521142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:26.521064    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:26.576527 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:24:26.576494    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:24:26.660422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:26.660340    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:28.096901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:28.096871    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.622515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:29.622477    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:31.019953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:31.019917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:31.736531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:31.636418    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:31.736952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:31.716780    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"]
Oct 30 21:24:31.737392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:31.737359    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" podUID="f524c2ca-c055-443b-9d1e-594e6a761c2e" containerName="maas-api" containerID="cri-o://d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2" gracePeriod=30
Oct 30 21:24:31.882431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:31.882336    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:31.945894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:31.940965    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" podUID="f524c2ca-c055-443b-9d1e-594e6a761c2e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.111:8443/health\": dial tcp 10.133.0.111:8443: connect: connection refused"
Oct 30 21:24:32.354302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.350552    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.685189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.685144    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.701595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.701088    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:24:32.799475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.799430    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs\") pod \"f524c2ca-c055-443b-9d1e-594e6a761c2e\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") "
Oct 30 21:24:32.799692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.799511    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca\") pod \"f524c2ca-c055-443b-9d1e-594e6a761c2e\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") "
Oct 30 21:24:32.799692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.799551    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls\") pod \"f524c2ca-c055-443b-9d1e-594e6a761c2e\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") "
Oct 30 21:24:32.799810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.799706    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj\" (UniqueName: \"kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj\") pod \"f524c2ca-c055-443b-9d1e-594e6a761c2e\" (UID: \"f524c2ca-c055-443b-9d1e-594e6a761c2e\") "
Oct 30 21:24:32.799919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.799870    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca" pod "f524c2ca-c055-443b-9d1e-594e6a761c2e" (UID: "f524c2ca-c055-443b-9d1e-594e6a761c2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:32.800573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.800545    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f524c2ca-c055-443b-9d1e-594e6a761c2e-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:32.802307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.802275    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj" pod "f524c2ca-c055-443b-9d1e-594e6a761c2e" (UID: "f524c2ca-c055-443b-9d1e-594e6a761c2e"). InnerVolumeSpecName "kube-api-access-vwphj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:32.802524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.802499    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs" pod "f524c2ca-c055-443b-9d1e-594e6a761c2e" (UID: "f524c2ca-c055-443b-9d1e-594e6a761c2e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:32.802601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.802538    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls" pod "f524c2ca-c055-443b-9d1e-594e6a761c2e" (UID: "f524c2ca-c055-443b-9d1e-594e6a761c2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:32.824986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.824947    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.882031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.881993    4549 generic.go:356] "Generic (PLEG): container finished" podID="f524c2ca-c055-443b-9d1e-594e6a761c2e" containerID="d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2" exitCode=0
Oct 30 21:24:32.882204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.882038    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"
Oct 30 21:24:32.882204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.882077    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" event={"ID":"f524c2ca-c055-443b-9d1e-594e6a761c2e","Type":"ContainerDied","Data":"d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2"}
Oct 30 21:24:32.882204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.882133    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8" event={"ID":"f524c2ca-c055-443b-9d1e-594e6a761c2e","Type":"ContainerDied","Data":"6bd3d46592521a90e1814d944558c37f14ddad53a5fe05741f5cf6a1611e925e"}
Oct 30 21:24:32.882204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.882161    4549 scope.go:122] "RemoveContainer" containerID="d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2"
Oct 30 21:24:32.894218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.894199    4549 scope.go:122] "RemoveContainer" containerID="d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2"
Oct 30 21:24:32.894513 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:24:32.894495    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2\": container with ID starting with d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2 not found: ID does not exist" containerID="d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2"
Oct 30 21:24:32.894563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.894521    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2"} err="failed to get container status \"d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2\": rpc error: code = NotFound desc = could not find container \"d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2\": container with ID starting with d7d0c9879a715d5c42a33cd4a9e78d3a48e866a4d025e1230b9edee02dd500f2 not found: ID does not exist"
Oct 30 21:24:32.912542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.912450    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:32.912542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.912479    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f524c2ca-c055-443b-9d1e-594e6a761c2e-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:32.912542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.912489    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwphj\" (UniqueName: \"kubernetes.io/projected/f524c2ca-c055-443b-9d1e-594e6a761c2e-kube-api-access-vwphj\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:32.956481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.956384    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.968862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.966513    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"]
Oct 30 21:24:32.976438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:32.974059    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0e7510-77f7bf9795-95ll8"]
Oct 30 21:24:33.336931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:33.336893    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:33.347380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:33.347354    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f524c2ca-c055-443b-9d1e-594e6a761c2e" path="/var/lib/kubelet/pods/f524c2ca-c055-443b-9d1e-594e6a761c2e/volumes"
Oct 30 21:24:34.497917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:34.497878    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:35.938602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:35.938566    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:37.430371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:37.430332    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:38.271597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.271514    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:38.489876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.485362    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:38.758543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.758504    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"]
Oct 30 21:24:38.758771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.758745    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy" containerID="cri-o://7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49" gracePeriod=30
Oct 30 21:24:38.765946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.765912    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"]
Oct 30 21:24:38.766164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.766135    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy" containerID="cri-o://4e9e4a6722b64b1721016cb3fcbb7394a11defe1f99199029706326c2b8331a6" gracePeriod=30
Oct 30 21:24:38.776740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:38.776479    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:39.618595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:39.618560    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.210670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:40.210639    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.376894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:40.376812    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.614106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:40.614071    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:41.659855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:41.652420    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:44.255856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.221719    4549 generic.go:356] "Generic (PLEG): container finished" podID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerID="7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49" exitCode=0
Oct 30 21:24:44.255856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.221887    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" event={"ID":"aa530f5f-e62a-4321-830d-19f4b987dad8","Type":"ContainerDied","Data":"7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49"}
Oct 30 21:24:44.292857 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:24:44.279293    4549 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa530f5f_e62a_4321_830d_19f4b987dad8.slice/crio-conmon-7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddf7906_05f9_40d5_a863_28b79fedcf3f.slice/crio-4e9e4a6722b64b1721016cb3fcbb7394a11defe1f99199029706326c2b8331a6.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa530f5f_e62a_4321_830d_19f4b987dad8.slice/crio-7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:24:44.304856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.300215    4549 generic.go:356] "Generic (PLEG): container finished" podID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerID="4e9e4a6722b64b1721016cb3fcbb7394a11defe1f99199029706326c2b8331a6" exitCode=0
Oct 30 21:24:44.304856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.300412    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" event={"ID":"6ddf7906-05f9-40d5-a863-28b79fedcf3f","Type":"ContainerDied","Data":"4e9e4a6722b64b1721016cb3fcbb7394a11defe1f99199029706326c2b8331a6"}
Oct 30 21:24:44.798381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.796983    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:24:44.798381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.797105    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:44.999320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:44.998941    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068150    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068229    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068276    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068327    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068496    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068594    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068634    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068685    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.070260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.068880    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo\") pod \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\" (UID: \"6ddf7906-05f9-40d5-a863-28b79fedcf3f\") "
Oct 30 21:24:45.079655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.079275    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:45.136919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.107997    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:45.136919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.113973    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:45.136919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.117985    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:45.136919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.123985    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:45.136919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.125996    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:45.393805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.341406    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:24:45.393805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.341677    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "kube-api-access-bhmmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:45.393805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.341753    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:45.578859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.578475    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token" pod "6ddf7906-05f9-40d5-a863-28b79fedcf3f" (UID: "6ddf7906-05f9-40d5-a863-28b79fedcf3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:45.690412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.690304    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:45.860855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.852156    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861279    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861312    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861331    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861353    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861369    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhmmq\" (UniqueName: \"kubernetes.io/projected/6ddf7906-05f9-40d5-a863-28b79fedcf3f-kube-api-access-bhmmq\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861416    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861432    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.861452    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddf7906-05f9-40d5-a863-28b79fedcf3f-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:45.871973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.863360    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.938929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:45.937846    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:24:46.143854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.141553    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.143854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.142521    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:46.273857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.249653    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"
Oct 30 21:24:46.439917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.344198    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:46.439917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.344236    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:24:46.459860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.376296    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.550433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.550393    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.550779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.550756    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.551087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.551066    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.551279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.551260    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.551430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.551413    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.551688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.551671    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.551813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.551799    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.551991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.551975    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo\") pod \"aa530f5f-e62a-4321-830d-19f4b987dad8\" (UID: \"aa530f5f-e62a-4321-830d-19f4b987dad8\") "
Oct 30 21:24:46.580937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.405214    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:46.614467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.603720    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:46.614467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.603790    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:46.614467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.608033    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:46.628984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.628798    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:24:46.633869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.631614    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"
Oct 30 21:24:46.638894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.638815    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "kube-api-access-9lwrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:46.642818 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:24:46.642789    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.032s"
Oct 30 21:24:46.642962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.642928    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57" event={"ID":"aa530f5f-e62a-4321-830d-19f4b987dad8","Type":"ContainerDied","Data":"12a43d8112e33337747617026e1ee6e395dda59d64d202ba1a7e11f1bfd4b5fd"}
Oct 30 21:24:46.643030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.642973    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54" event={"ID":"6ddf7906-05f9-40d5-a863-28b79fedcf3f","Type":"ContainerDied","Data":"adaba52dbd367ecc257d7b932a4d2ce42d33c0dea7e3748e7a21d69fb0027b25"}
Oct 30 21:24:46.643089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.643046    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:24:46.643089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.643084    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:24:46.646170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.643580    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:46.646170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.643612    4549 scope.go:122] "RemoveContainer" containerID="7ef359f3a74f9804427b38c5243ceed78db3c33bcf27545a2f365e6dd6025a49"
Oct 30 21:24:46.646170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.644652    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:46.646754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.646657    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token" pod "aa530f5f-e62a-4321-830d-19f4b987dad8" (UID: "aa530f5f-e62a-4321-830d-19f4b987dad8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:46.654866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654841    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.654866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654868    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654878    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lwrp\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-kube-api-access-9lwrp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654888    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654896    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654903    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654911    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654919    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa530f5f-e62a-4321-830d-19f4b987dad8-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.655098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.654929    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa530f5f-e62a-4321-830d-19f4b987dad8-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:24:46.692141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.688082    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.776182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.768809    4549 scope.go:122] "RemoveContainer" containerID="4e9e4a6722b64b1721016cb3fcbb7394a11defe1f99199029706326c2b8331a6"
Oct 30 21:24:46.821558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.821496    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"]
Oct 30 21:24:46.835921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.835144    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54"]
Oct 30 21:24:46.932853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.932759    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"]
Oct 30 21:24:46.964210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:46.964042    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57"]
Oct 30 21:24:47.346612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:47.346578    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" path="/var/lib/kubelet/pods/6ddf7906-05f9-40d5-a863-28b79fedcf3f/volumes"
Oct 30 21:24:47.347425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:47.347402    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" path="/var/lib/kubelet/pods/aa530f5f-e62a-4321-830d-19f4b987dad8/volumes"
Oct 30 21:24:47.710918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:47.710779    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.854294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:47.853823    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:49.626202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:49.626171    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.040092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:50.039961    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.282261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:50.277424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.542317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:51.542283    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.713693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:51.713653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.940442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:51.940355    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:53.434883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:53.434784    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:55.008789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:55.008746    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:56.685919 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:24:56.684977    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:24:56.944561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:56.944482    4549 generic.go:356] "Generic (PLEG): container finished" podID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerID="0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0" exitCode=0
Oct 30 21:24:56.944723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:56.944553    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" event={"ID":"0080a530-5406-4ecb-8fcc-ad4fa5fb0793","Type":"ContainerDied","Data":"0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0"}
Oct 30 21:24:56.945158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:56.945139    4549 scope.go:122] "RemoveContainer" containerID="0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0"
Oct 30 21:24:56.947452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:56.947417    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:56.947541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:56.947512    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:56.947581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:56.947563    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:24:57.050445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:57.050399    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:57.955437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:57.955396    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" event={"ID":"0080a530-5406-4ecb-8fcc-ad4fa5fb0793","Type":"ContainerStarted","Data":"4cb77a12939b4548a98d5b7f3386c6503eb4edc851b29ac9678434ffe6186a89"}
Oct 30 21:24:58.697714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:58.697440    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:58.729867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:58.720256    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:24:58.731027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:58.730952    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 21:24:58.731348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:58.731280    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 21:24:59.720692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:59.720653    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 21:24:59.721278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:24:59.720722    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 21:25:00.718506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:00.718474    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:00.720363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:00.720336    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 21:25:00.720499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:00.720391    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 21:25:00.948975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:00.948936    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:25:00.949375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:00.949003    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:25:01.720015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:01.719973    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 21:25:01.720192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:01.720046    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 21:25:01.739663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:01.739634    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:25:02.369441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:02.369338    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:02.719791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:02.719750    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 21:25:02.720067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:02.719855    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 21:25:03.020589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.020500    4549 generic.go:356] "Generic (PLEG): container finished" podID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerID="b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c" exitCode=0
Oct 30 21:25:03.020741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.020675    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" event={"ID":"0183f5a1-7bd4-4190-acb0-8dbe84649986","Type":"ContainerDied","Data":"b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"}
Oct 30 21:25:03.021279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.021257    4549 scope.go:122] "RemoveContainer" containerID="b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"
Oct 30 21:25:03.023384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.023348    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:03.023466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.023426    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:03.023510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.023481    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:03.720092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.720054    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 21:25:03.720590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.720118    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 21:25:03.833475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.833438    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-c12bc9"]
Oct 30 21:25:03.840253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.840218    4549 cpu_manager.go:414] "containerMap: removing container" podUID="f524c2ca-c055-443b-9d1e-594e6a761c2e" containerName="maas-api"
Oct 30 21:25:03.840466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.840451    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f524c2ca-c055-443b-9d1e-594e6a761c2e" containerName="maas-api"
Oct 30 21:25:03.840571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.840561    4549 cpu_manager.go:414] "containerMap: removing container" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy"
Oct 30 21:25:03.840650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.840640    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy"
Oct 30 21:25:03.840744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.840734    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy"
Oct 30 21:25:03.840821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.840812    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy"
Oct 30 21:25:03.841226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.841210    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="f524c2ca-c055-443b-9d1e-594e6a761c2e" containerName="maas-api"
Oct 30 21:25:03.841326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.841317    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa530f5f-e62a-4321-830d-19f4b987dad8" containerName="istio-proxy"
Oct 30 21:25:03.841380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.841374    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ddf7906-05f9-40d5-a863-28b79fedcf3f" containerName="istio-proxy"
Oct 30 21:25:03.844699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.844674    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:03.858948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.858902    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-c12bc9"]
Oct 30 21:25:03.866231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:03.866198    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxp2m\""
Oct 30 21:25:04.025333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.025238    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:04.032191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.032161    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" event={"ID":"0183f5a1-7bd4-4190-acb0-8dbe84649986","Type":"ContainerStarted","Data":"6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e"}
Oct 30 21:25:04.033936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.033913    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67rv\" (UniqueName: \"kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv\") pod \"test-curl-7404-c12bc9\" (UID: \"e45e79a3-f835-4764-8778-21533e635030\") " pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:04.138873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.137157    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67rv\" (UniqueName: \"kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv\") pod \"test-curl-7404-c12bc9\" (UID: \"e45e79a3-f835-4764-8778-21533e635030\") " pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:04.186855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.186774    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67rv\" (UniqueName: \"kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv\") pod \"test-curl-7404-c12bc9\" (UID: \"e45e79a3-f835-4764-8778-21533e635030\") " pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:04.243789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.243054    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:04.269307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.269267    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:25:04.269610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.269363    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:25:04.269799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.269768    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 21:25:04.633113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.633082    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-c12bc9"]
Oct 30 21:25:04.634096 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:25:04.634046    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode45e79a3_f835_4764_8778_21533e635030.slice/crio-0da4aa0be5e1032643d0c7e8ececbb36ad1366de59ac8717b3c4113c31fc7577 WatchSource:0}: Error finding container 0da4aa0be5e1032643d0c7e8ececbb36ad1366de59ac8717b3c4113c31fc7577: Status 404 returned error can't find the container with id 0da4aa0be5e1032643d0c7e8ececbb36ad1366de59ac8717b3c4113c31fc7577
Oct 30 21:25:04.817947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.817911    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:25:04.819510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:04.819487    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:25:05.042136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.042040    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-c12bc9" event={"ID":"e45e79a3-f835-4764-8778-21533e635030","Type":"ContainerStarted","Data":"e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd"}
Oct 30 21:25:05.042136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.042092    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-c12bc9" event={"ID":"e45e79a3-f835-4764-8778-21533e635030","Type":"ContainerStarted","Data":"0da4aa0be5e1032643d0c7e8ececbb36ad1366de59ac8717b3c4113c31fc7577"}
Oct 30 21:25:05.096504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.096430    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7404-c12bc9" podStartSLOduration=2.096407761 podStartE2EDuration="2.096407761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:05.094040266 +0000 UTC m=+1487.868440208" watchObservedRunningTime="2026-10-30 21:25:05.096407761 +0000 UTC m=+1487.870807701"
Oct 30 21:25:05.268712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.268680    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:25:05.268973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.268748    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 21:25:05.440901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.440867    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:05.759581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:05.759475    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:06.149854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:06.149804    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-c12bc9"]
Oct 30 21:25:06.155265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:06.155230    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-c12bc9"]
Oct 30 21:25:06.268408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:06.268376    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:25:06.268598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:06.268430    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 21:25:06.695377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:06.695314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:07.016430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:07.016297    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:07.057108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:07.057065    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7404-c12bc9" podUID="e45e79a3-f835-4764-8778-21533e635030" containerName="test-curl-7404-c12bc9" containerID="cri-o://e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd" gracePeriod=2
Oct 30 21:25:07.081111 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:25:07.079107    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7404-c12bc9\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="e45e79a3-f835-4764-8778-21533e635030" pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:07.208021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:07.207974    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:07.286136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:07.285998    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:25:07.289960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:07.289910    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 21:25:07.479659 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:25:07.479620    4549 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7404-c12bc9\" is forbidden: User \"system:node:ip-10-0-140-179.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-179.ec2.internal' and this object" podUID="e45e79a3-f835-4764-8778-21533e635030" pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:07.651802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:07.651766    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:08.232647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:08.232607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:08.268022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:08.267981    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:25:08.268314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:08.268285    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 21:25:09.268059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.268029    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:25:09.268558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.268075    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 21:25:09.335637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.335608    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:09.463574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.463541    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv\" (UniqueName: \"kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv\") pod \"e45e79a3-f835-4764-8778-21533e635030\" (UID: \"e45e79a3-f835-4764-8778-21533e635030\") "
Oct 30 21:25:09.466061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.466030    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv" pod "e45e79a3-f835-4764-8778-21533e635030" (UID: "e45e79a3-f835-4764-8778-21533e635030"). InnerVolumeSpecName "kube-api-access-s67rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:09.566453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.566422    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s67rv\" (UniqueName: \"kubernetes.io/projected/e45e79a3-f835-4764-8778-21533e635030-kube-api-access-s67rv\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:09.897434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:09.897347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:10.115922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.114145    4549 generic.go:356] "Generic (PLEG): container finished" podID="e45e79a3-f835-4764-8778-21533e635030" containerID="e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd" exitCode=137
Oct 30 21:25:10.115922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.114211    4549 scope.go:122] "RemoveContainer" containerID="e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd"
Oct 30 21:25:10.115922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.114223    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-c12bc9"
Oct 30 21:25:10.130972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.130951    4549 scope.go:122] "RemoveContainer" containerID="e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd"
Oct 30 21:25:10.131238 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:25:10.131217    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd\": container with ID starting with e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd not found: ID does not exist" containerID="e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd"
Oct 30 21:25:10.131325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.131250    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd"} err="failed to get container status \"e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd\": rpc error: code = NotFound desc = could not find container \"e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd\": container with ID starting with e1321741e6c3f72a044b6679d971208df19256cf13f3441f9f9d8755f7234cfd not found: ID does not exist"
Oct 30 21:25:10.273547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.273463    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:25:10.274059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.274036    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:25:10.274816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.274792    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:25:10.654501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.654435    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:10.937448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:10.937361    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:11.345548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:11.345511    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e45e79a3-f835-4764-8778-21533e635030" path="/var/lib/kubelet/pods/e45e79a3-f835-4764-8778-21533e635030/volumes"
Oct 30 21:25:11.932230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:11.932190    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:13.422646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.422609    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:13.467510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.467476    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"]
Oct 30 21:25:13.468596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.468575    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e45e79a3-f835-4764-8778-21533e635030" containerName="test-curl-7404-c12bc9"
Oct 30 21:25:13.468596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.468595    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45e79a3-f835-4764-8778-21533e635030" containerName="test-curl-7404-c12bc9"
Oct 30 21:25:13.468733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.468719    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45e79a3-f835-4764-8778-21533e635030" containerName="test-curl-7404-c12bc9"
Oct 30 21:25:13.537057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.537026    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:13.540096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.540067    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"]
Oct 30 21:25:13.558004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.557978    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:13.624858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.611442    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-1eab65\""
Oct 30 21:25:13.838853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.829959    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rpr7\" (UniqueName: \"kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7\") pod \"payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:13.838853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.830048    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:13.968870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.956550    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rpr7\" (UniqueName: \"kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7\") pod \"payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:13.969266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.969233    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:13.988857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:13.970457    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:14.084419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:14.084387    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rpr7\" (UniqueName: \"kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7\") pod \"payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:14.152855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:14.152748    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:14.430313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:14.430190    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:14.640871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:14.640821    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"]
Oct 30 21:25:14.678543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:14.678502    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:14.883098 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:25:14.883047    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4483c37_9d44_46e4_9266_8dbb05824159.slice/crio-03ae543e1229d2ce7e3cf4ef0bf99f8db8c8361ee3db7e44febf9c9c73132aa9 WatchSource:0}: Error finding container 03ae543e1229d2ce7e3cf4ef0bf99f8db8c8361ee3db7e44febf9c9c73132aa9: Status 404 returned error can't find the container with id 03ae543e1229d2ce7e3cf4ef0bf99f8db8c8361ee3db7e44febf9c9c73132aa9
Oct 30 21:25:15.121901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.120909    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:15.230506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.230462    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" event={"ID":"c4483c37-9d44-46e4-9266-8dbb05824159","Type":"ContainerStarted","Data":"01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a"}
Oct 30 21:25:15.230680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.230537    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" event={"ID":"c4483c37-9d44-46e4-9266-8dbb05824159","Type":"ContainerStarted","Data":"03ae543e1229d2ce7e3cf4ef0bf99f8db8c8361ee3db7e44febf9c9c73132aa9"}
Oct 30 21:25:15.230680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.230568    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:15.290857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.283314    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" podStartSLOduration=2.283290293 podStartE2EDuration="2.283290293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:15.273271692 +0000 UTC m=+1498.047671644" watchObservedRunningTime="2026-10-30 21:25:15.283290293 +0000 UTC m=+1498.057690234"
Oct 30 21:25:15.751628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.751592    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"]
Oct 30 21:25:15.781204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.781164    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:15.795766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:15.794048    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"]
Oct 30 21:25:16.041913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.002240    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-093daa-openshift-default-dockercfg-wbwdl\""
Oct 30 21:25:16.097416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.097096    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.124975    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125121    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125195    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125226    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125310    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fncmg\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125385    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125454    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125522    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.181859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.125575    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.482864    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.482977    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483018    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483110    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fncmg\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483311    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483400    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483481    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483538    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.488929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.483723    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.502050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.501990    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.521852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.506923    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.526565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.516954    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.526778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.522966    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.526938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.523443    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.527033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.523870    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.583852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.558365    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.642700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.642665    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.643795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.643693    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fncmg\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg\") pod \"e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") " pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.708859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.704170    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:16.708859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.707773    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:16.924933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:16.924852    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:17.131854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:17.129280    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:17.281910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:17.281696    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:17.966211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:17.965763    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:18.038926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.026076    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"]
Oct 30 21:25:18.054676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.054632    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:18.054971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.054941    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:18.055133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.055116    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:18.114916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.102919    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:18.158726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.158695    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:25:18.160905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.160882    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:25:18.764343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:18.764305    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:19.065569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:19.065534    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" event={"ID":"22f11dbc-7817-471f-b839-0b277254e60b","Type":"ContainerStarted","Data":"1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"}
Oct 30 21:25:19.066123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:19.065594    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" event={"ID":"22f11dbc-7817-471f-b839-0b277254e60b","Type":"ContainerStarted","Data":"4ddab8f3d7c3c113ad5b85ac4179af853f0cc375f8ced0324fb9a02eb3ec5221"}
Oct 30 21:25:19.155452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:19.155404    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podStartSLOduration=4.155390354 podStartE2EDuration="4.155390354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:19.140317784 +0000 UTC m=+1501.914717762" watchObservedRunningTime="2026-10-30 21:25:19.155390354 +0000 UTC m=+1501.929790353"
Oct 30 21:25:19.707260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:19.707228    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:19.715112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:19.715087    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:20.076746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:20.076714    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:20.078509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:20.078484    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:25:20.428495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:20.428401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:21.504038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:21.503998    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:25:21.504602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:21.504093    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:25:21.732917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:21.732782    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:21.951505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:21.951451    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:22.357376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:22.342950    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.305007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.304967    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.379029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.378996    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"]
Oct 30 21:25:23.405462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.405428    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"]
Oct 30 21:25:23.405678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.405521    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:23.425252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.425226    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-093daa-dockercfg-2gf76\""
Oct 30 21:25:23.437135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.437108    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-093daa\""
Oct 30 21:25:23.487198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.487166    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.504519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.494028    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"]
Oct 30 21:25:23.699985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.699959    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:23.734927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.706291    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"]
Oct 30 21:25:23.878712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.715377    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z2p5\" (UniqueName: \"kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5\") pod \"payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") " pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:23.878712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.878445    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") " pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:23.982536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.980504    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:23.982536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.980640    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvv77\" (UniqueName: \"kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77\") pod \"payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:23.982536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.980690    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") " pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:23.982536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.980862    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z2p5\" (UniqueName: \"kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5\") pod \"payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") " pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:23.982536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:23.982314    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") " pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:24.010436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.010401    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z2p5\" (UniqueName: \"kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5\") pod \"payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") " pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:24.036386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.036349    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:24.083728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.083677    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvv77\" (UniqueName: \"kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77\") pod \"payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:24.084102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.084077    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:24.085138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.085085    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:24.120629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.119451    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvv77\" (UniqueName: \"kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77\") pod \"payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:24.281544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.281446    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:24.408334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.394956    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"]
Oct 30 21:25:24.418507 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:25:24.418141    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2dc276_80c4_48f1_9a81_2e5197378916.slice/crio-240b52799c338e3d8052a7655584cb1b761506999832fc4d19909d10df363633 WatchSource:0}: Error finding container 240b52799c338e3d8052a7655584cb1b761506999832fc4d19909d10df363633: Status 404 returned error can't find the container with id 240b52799c338e3d8052a7655584cb1b761506999832fc4d19909d10df363633
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.629792    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"]
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:24.632862    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.131218    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" event={"ID":"87fda6f5-520b-4855-a3d6-de68f43bf1c8","Type":"ContainerStarted","Data":"7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898"}
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.131279    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" event={"ID":"87fda6f5-520b-4855-a3d6-de68f43bf1c8","Type":"ContainerStarted","Data":"98ff286478b324b7c68b680011e2b368af73361243e04fa51a8fa1e8f357a9c0"}
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.131308    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.138220    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" event={"ID":"0f2dc276-80c4-48f1-9a81-2e5197378916","Type":"ContainerStarted","Data":"5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab"}
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.138285    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" event={"ID":"0f2dc276-80c4-48f1-9a81-2e5197378916","Type":"ContainerStarted","Data":"240b52799c338e3d8052a7655584cb1b761506999832fc4d19909d10df363633"}
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.138315    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:25.190283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.177214    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" podStartSLOduration=2.177193693 podStartE2EDuration="2.177193693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:25.173368685 +0000 UTC m=+1507.947768636" watchObservedRunningTime="2026-10-30 21:25:25.177193693 +0000 UTC m=+1507.951593632"
Oct 30 21:25:25.216585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:25.216523    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" podStartSLOduration=2.216505491 podStartE2EDuration="2.216505491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:25.214249854 +0000 UTC m=+1507.988649792" watchObservedRunningTime="2026-10-30 21:25:25.216505491 +0000 UTC m=+1507.990905430"
Oct 30 21:25:26.241173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.240529    4549 generic.go:356] "Generic (PLEG): container finished" podID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerID="deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085" exitCode=0
Oct 30 21:25:26.241173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.240685    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" event={"ID":"7d18cb2f-8a61-494c-9cae-46768da2d8eb","Type":"ContainerDied","Data":"deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"}
Oct 30 21:25:26.241761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.241490    4549 scope.go:122] "RemoveContainer" containerID="deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"
Oct 30 21:25:26.253576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.253525    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:26.269510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.269465    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:26.269791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.269768    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:26.383708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.373591    4549 generic.go:356] "Generic (PLEG): container finished" podID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerID="e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3" exitCode=0
Oct 30 21:25:26.383708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.375094    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" event={"ID":"2be14cc1-2729-4b2a-a37f-b01dd0b27a84","Type":"ContainerDied","Data":"e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"}
Oct 30 21:25:26.383708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.375683    4549 scope.go:122] "RemoveContainer" containerID="e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"
Oct 30 21:25:26.505149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.504979    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:25:26.506356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.506318    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:26.506504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.506404    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:26.506504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.506456    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:26.527340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:26.527309    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:26.609629 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:25:26.609485    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:25:27.393470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:27.393428    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" event={"ID":"2be14cc1-2729-4b2a-a37f-b01dd0b27a84","Type":"ContainerStarted","Data":"06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0"}
Oct 30 21:25:27.398353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:27.398320    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" event={"ID":"7d18cb2f-8a61-494c-9cae-46768da2d8eb","Type":"ContainerStarted","Data":"13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60"}
Oct 30 21:25:28.155201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.155163    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:25:28.155390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.155369    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:25:28.155441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.155418    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:25:28.330362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.330328    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:25:28.332920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.332888    4549 patch_prober.go:29] interesting pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:25:28.333071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.332949    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:25:28.431128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.431036    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:28.956193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:28.956157    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:29.155157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:29.155113    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:25:29.155478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:29.155433    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:25:29.329257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:29.329221    4549 patch_prober.go:29] interesting pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:25:29.329461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:29.329273    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:25:30.155754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:30.155713    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:25:30.156190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:30.155797    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:25:30.329498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:30.329458    4549 patch_prober.go:29] interesting pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:25:30.329795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:30.329760    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:25:30.421721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:30.421629    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:31.098899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:31.092586    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:25:31.155322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:31.155289    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:25:31.155543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:31.155374    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:25:31.328727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:31.328691    4549 patch_prober.go:29] interesting pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:25:31.333040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:31.332985    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:25:32.155937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:32.155904    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:25:32.156185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:32.156164    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:25:32.328375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:32.328343    4549 patch_prober.go:29] interesting pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:25:32.328603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:32.328395    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:25:32.562905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:32.559661    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:33.219606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:33.219571    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:25:33.220965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:33.220936    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:25:33.222403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:33.222378    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:25:33.334274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:33.334241    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:25:33.335325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:33.335297    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:25:34.219275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:34.219242    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.187606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:35.187567    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.345430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:35.345394    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.488900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:35.487753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.896038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:35.895997    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:36.378036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:36.378005    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:25:36.391144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:36.391117    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:25:37.117508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:37.117463    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:37.627212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:37.627175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:38.048213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.048117    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"]
Oct 30 21:25:38.052373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.052346    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.073671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.073643    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-79c24a-openshift-default-dockercfg-k6ggn\""
Oct 30 21:25:38.094905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.094866    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"]
Oct 30 21:25:38.131422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.131388    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:38.259860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.238488    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.304031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.303933    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmgs2\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.304451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.304422    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.304624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.304608    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.304764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.304750    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.332358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.332321    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.332716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.332691    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.333013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.332990    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.333233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.333216    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.538890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.504396    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.539311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.539269    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.539523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.539504    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.561375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.540431    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.561613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.555962    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.561914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.561889    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.562177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.562156    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.562348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.562330    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.562552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.562533    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.562729 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.562705    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.562868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.562850    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.563007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.562991    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmgs2\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.563639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.563608    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.563906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.563884    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.564098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.563022    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.570142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.570108    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.597575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.597134    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmgs2\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.598622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.598565    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.675476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.675391    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:38.897303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:38.897144    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:39.021229 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:25:39.021197    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d79a5cf_d13a_4723_9ea2_517205e31827.slice/crio-36e482ecfdeeddde6f0fbeec9a4e0aca2fb1760a824b4d82cddbdc3e28f29c2b WatchSource:0}: Error finding container 36e482ecfdeeddde6f0fbeec9a4e0aca2fb1760a824b4d82cddbdc3e28f29c2b: Status 404 returned error can't find the container with id 36e482ecfdeeddde6f0fbeec9a4e0aca2fb1760a824b4d82cddbdc3e28f29c2b
Oct 30 21:25:39.024873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.024814    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:39.024996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.024912    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:39.024996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.024944    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:25:39.028550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.028518    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"]
Oct 30 21:25:39.598069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.598031    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" event={"ID":"6d79a5cf-d13a-4723-9ea2-517205e31827","Type":"ContainerStarted","Data":"e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"}
Oct 30 21:25:39.598259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.598099    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" event={"ID":"6d79a5cf-d13a-4723-9ea2-517205e31827","Type":"ContainerStarted","Data":"36e482ecfdeeddde6f0fbeec9a4e0aca2fb1760a824b4d82cddbdc3e28f29c2b"}
Oct 30 21:25:39.660994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.660921    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podStartSLOduration=1.660899643 podStartE2EDuration="1.660899643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:39.6562512 +0000 UTC m=+1522.430651138" watchObservedRunningTime="2026-10-30 21:25:39.660899643 +0000 UTC m=+1522.435299596"
Oct 30 21:25:39.675539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:39.675506    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:40.728190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:40.728150    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:40.862194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:40.862151    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"]
Oct 30 21:25:41.105471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:41.105439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:41.223790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:41.223754    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:41.612996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:41.612963    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:41.613894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:41.613876    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:25:41.795312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:41.795273    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:41.929715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:41.926643    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:42.228292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:42.228186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:42.396030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:42.395997    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:44.062043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:44.061995    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:44.230228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:44.230183    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:44.403154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:44.403087    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.528890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:46.528851    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:48.038667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:48.038631    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:48.187876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:48.187456    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.609357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:49.609311    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:51.236652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:51.236615    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:52.810845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:52.810787    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.237226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:54.237193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:55.702452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:55.702417    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:55.912393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:55.912326    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" containerID="cri-o://172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f" gracePeriod=45
Oct 30 21:25:56.258602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.258576    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:25:56.267935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.267906    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home\") pod \"263b186c-9d33-4e08-81b7-fa11f0731942\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") "
Oct 30 21:25:56.268115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.267988    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache\") pod \"263b186c-9d33-4e08-81b7-fa11f0731942\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") "
Oct 30 21:25:56.268115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268087    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6\" (UniqueName: \"kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6\") pod \"263b186c-9d33-4e08-81b7-fa11f0731942\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") "
Oct 30 21:25:56.268243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268168    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm\") pod \"263b186c-9d33-4e08-81b7-fa11f0731942\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") "
Oct 30 21:25:56.268243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268209    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home" pod "263b186c-9d33-4e08-81b7-fa11f0731942" (UID: "263b186c-9d33-4e08-81b7-fa11f0731942"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:56.268243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268235    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs\" (UniqueName: \"kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs\") pod \"263b186c-9d33-4e08-81b7-fa11f0731942\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") "
Oct 30 21:25:56.268376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268260    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache" pod "263b186c-9d33-4e08-81b7-fa11f0731942" (UID: "263b186c-9d33-4e08-81b7-fa11f0731942"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:56.268376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268283    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir\") pod \"263b186c-9d33-4e08-81b7-fa11f0731942\" (UID: \"263b186c-9d33-4e08-81b7-fa11f0731942\") "
Oct 30 21:25:56.268584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.268558    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir" pod "263b186c-9d33-4e08-81b7-fa11f0731942" (UID: "263b186c-9d33-4e08-81b7-fa11f0731942"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:56.269064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.269045    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:56.269064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.269065    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:56.269234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.269074    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:56.270576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.270547    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm" pod "263b186c-9d33-4e08-81b7-fa11f0731942" (UID: "263b186c-9d33-4e08-81b7-fa11f0731942"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:56.270576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.270553    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs" pod "263b186c-9d33-4e08-81b7-fa11f0731942" (UID: "263b186c-9d33-4e08-81b7-fa11f0731942"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:25:56.270716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.270640    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6" pod "263b186c-9d33-4e08-81b7-fa11f0731942" (UID: "263b186c-9d33-4e08-81b7-fa11f0731942"). InnerVolumeSpecName "kube-api-access-cl6j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:56.371633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.371552    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl6j6\" (UniqueName: \"kubernetes.io/projected/263b186c-9d33-4e08-81b7-fa11f0731942-kube-api-access-cl6j6\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:56.371633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.371587    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/263b186c-9d33-4e08-81b7-fa11f0731942-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:56.371633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.371602    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/263b186c-9d33-4e08-81b7-fa11f0731942-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:25:56.578529 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:25:56.578491    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:25:56.867510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.867481    4549 generic.go:356] "Generic (PLEG): container finished" podID="263b186c-9d33-4e08-81b7-fa11f0731942" containerID="172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f" exitCode=0
Oct 30 21:25:56.868149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.867661    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" event={"ID":"263b186c-9d33-4e08-81b7-fa11f0731942","Type":"ContainerDied","Data":"172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f"}
Oct 30 21:25:56.868248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.868181    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" event={"ID":"263b186c-9d33-4e08-81b7-fa11f0731942","Type":"ContainerDied","Data":"33939c95a5aebae2e86b514adba63c4493cc111812a75ab40d39420af2065a56"}
Oct 30 21:25:56.868248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.867757    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"
Oct 30 21:25:56.868248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.868210    4549 scope.go:122] "RemoveContainer" containerID="172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f"
Oct 30 21:25:56.884520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.884497    4549 scope.go:122] "RemoveContainer" containerID="172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f"
Oct 30 21:25:56.884908 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:25:56.884884    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f\": container with ID starting with 172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f not found: ID does not exist" containerID="172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f"
Oct 30 21:25:56.885017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.884918    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f"} err="failed to get container status \"172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f\": rpc error: code = NotFound desc = could not find container \"172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f\": container with ID starting with 172df7756823de60365e086200640838c3764cedef5b6a17fe0bf6b8eff05b1f not found: ID does not exist"
Oct 30 21:25:56.923788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.923753    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"]
Oct 30 21:25:56.935476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.935444    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9"]
Oct 30 21:25:56.978387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:56.978273    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.229649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:57.228950    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-c1da2183/e2e-lifecycle-model-c1da2183-kserve-cbc5498b9-sl7q9" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main" probeResult="failure" output="Get \"https://10.133.0.109:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:25:57.348097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:57.348061    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" path="/var/lib/kubelet/pods/263b186c-9d33-4e08-81b7-fa11f0731942/volumes"
Oct 30 21:25:57.469851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:57.468763    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:58.972534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:25:58.972499    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:00.326299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:00.326268    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:00.665664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:00.665578    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:00.817699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:00.817654    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:01.007606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:01.007507    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:01.602890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:01.597191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:02.028498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:02.028385    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:02.174005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:02.173970    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:02.301904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:02.300744    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:02.712757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:02.712724    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:03.930240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:03.930204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.424528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:05.424493    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.902910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:06.902872    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:07.191403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:07.191317    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.361938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:08.361900    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.490873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:08.490654    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.642997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:08.642914    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"]
Oct 30 21:26:08.643233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:08.643203    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" containerID="cri-o://6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e" gracePeriod=30
Oct 30 21:26:08.810236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:08.768426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:09.530994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.530956    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:09.679057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.679023    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:09.817261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.817222    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:09.936671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.936634    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"]
Oct 30 21:26:09.937906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.937880    4549 cpu_manager.go:414] "containerMap: removing container" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main"
Oct 30 21:26:09.937906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.937906    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main"
Oct 30 21:26:09.938166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.938150    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="263b186c-9d33-4e08-81b7-fa11f0731942" containerName="main"
Oct 30 21:26:09.941965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.941942    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:09.968820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.968792    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"]
Oct 30 21:26:09.974420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:09.974392    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-79e911-openshift-default-dockercfg-66945\""
Oct 30 21:26:10.014052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.013996    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.019982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.019406    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"]
Oct 30 21:26:10.019982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.019639    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy" containerID="cri-o://fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf" gracePeriod=30
Oct 30 21:26:10.047484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047449    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.047689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047584    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p2wx\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.047689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047622    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.047689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047654    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.047878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047798    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.047878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047863    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.048029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.047904    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.048029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.048009    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.048141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.048056    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.214918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.195734    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.216019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.215977    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.216166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216083    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p2wx\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.216166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216114    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.216166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216146    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216310    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216360    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216400    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216513    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216553    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.216693    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.217129    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.217457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.217360    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.220853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.217601    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.245420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.221610    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.260860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.246155    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.280854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.264090    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.280854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.272297    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p2wx\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.281590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.281565    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token\") pod \"e2e-inf-a-79e911-openshift-default-67558899b7-2b97d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") " pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.296850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.286561    4549 patch_prober.go:29] interesting pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:26:10.296850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.286623    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:26:10.376900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.359562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.569908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.564818    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:10.866685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.866595    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.989949 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:10.989903    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ccf846_20db_4a8d_874e_428b2a90d59d.slice/crio-3c0ec9a15b0970fc8b40937e91638a80aa4c8d698be7cd6d7881f86b3269a887 WatchSource:0}: Error finding container 3c0ec9a15b0970fc8b40937e91638a80aa4c8d698be7cd6d7881f86b3269a887: Status 404 returned error can't find the container with id 3c0ec9a15b0970fc8b40937e91638a80aa4c8d698be7cd6d7881f86b3269a887
Oct 30 21:26:10.990406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.990381    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"]
Oct 30 21:26:10.993058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.993029    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:10.993139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.993093    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:10.993139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:10.993125    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:11.328270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.328234    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" event={"ID":"e6ccf846-20db-4a8d-874e-428b2a90d59d","Type":"ContainerStarted","Data":"108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"}
Oct 30 21:26:11.328504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.328299    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" event={"ID":"e6ccf846-20db-4a8d-874e-428b2a90d59d","Type":"ContainerStarted","Data":"3c0ec9a15b0970fc8b40937e91638a80aa4c8d698be7cd6d7881f86b3269a887"}
Oct 30 21:26:11.400687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.400622    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podStartSLOduration=2.400599161 podStartE2EDuration="2.400599161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:11.396307589 +0000 UTC m=+1554.170707529" watchObservedRunningTime="2026-10-30 21:26:11.400599161 +0000 UTC m=+1554.174999104"
Oct 30 21:26:11.493061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.493016    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:11.567216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.567190    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:11.667854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.653658    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:11.837908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:11.836851    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:12.574209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:12.574177    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:13.349218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:13.349179    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:13.350026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:13.350000    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:26:13.710617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:13.710586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:13.955213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:13.954584    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.039783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.039755    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:26:14.092505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092470    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.092505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092522    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.092798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092598    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.092798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092648    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.092798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092675    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.092798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092762    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.092798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092791    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.093095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092913    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:14.093095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.092964    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.093095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.093052    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token\") pod \"0183f5a1-7bd4-4190-acb0-8dbe84649986\" (UID: \"0183f5a1-7bd4-4190-acb0-8dbe84649986\") "
Oct 30 21:26:14.093261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.093139    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:14.093261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.093233    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:14.093373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.093337    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:14.093718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.093518    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:14.094019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.093994    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.094121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.094027    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0183f5a1-7bd4-4190-acb0-8dbe84649986-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.094121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.094043    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.094121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.094067    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.094287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.094156    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.095692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.095664    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:14.095896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.095808    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:14.096602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.096579    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:14.097977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.097953    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt" pod "0183f5a1-7bd4-4190-acb0-8dbe84649986" (UID: "0183f5a1-7bd4-4190-acb0-8dbe84649986"). InnerVolumeSpecName "kube-api-access-tzqxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:14.124918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.124821    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.196079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.195978    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.196079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.196012    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.196079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.196025    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0183f5a1-7bd4-4190-acb0-8dbe84649986-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.196079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.196036    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzqxt\" (UniqueName: \"kubernetes.io/projected/0183f5a1-7bd4-4190-acb0-8dbe84649986-kube-api-access-tzqxt\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:14.357339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.357301    4549 generic.go:356] "Generic (PLEG): container finished" podID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerID="6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e" exitCode=0
Oct 30 21:26:14.357894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.357355    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"
Oct 30 21:26:14.357894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.357378    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" event={"ID":"0183f5a1-7bd4-4190-acb0-8dbe84649986","Type":"ContainerDied","Data":"6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e"}
Oct 30 21:26:14.357894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.357445    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2" event={"ID":"0183f5a1-7bd4-4190-acb0-8dbe84649986","Type":"ContainerDied","Data":"78ced1b09e0f37b7eeb7933e2fdf45e3ee472232b4082b8b1fdb8fa36fb72ad4"}
Oct 30 21:26:14.357894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.357475    4549 scope.go:122] "RemoveContainer" containerID="6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e"
Oct 30 21:26:14.370324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.370303    4549 scope.go:122] "RemoveContainer" containerID="b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"
Oct 30 21:26:14.381312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.381275    4549 scope.go:122] "RemoveContainer" containerID="6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e"
Oct 30 21:26:14.381604 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:14.381586    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e\": container with ID starting with 6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e not found: ID does not exist" containerID="6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e"
Oct 30 21:26:14.381664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.381615    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e"} err="failed to get container status \"6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e\": rpc error: code = NotFound desc = could not find container \"6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e\": container with ID starting with 6f26bf7a8a9b7953d30cecbee19d56335d16884b1ef27edc623ffaa1c2567d9e not found: ID does not exist"
Oct 30 21:26:14.381664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.381633    4549 scope.go:122] "RemoveContainer" containerID="b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"
Oct 30 21:26:14.381978 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:14.381958    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c\": container with ID starting with b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c not found: ID does not exist" containerID="b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"
Oct 30 21:26:14.382040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.381987    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c"} err="failed to get container status \"b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c\": rpc error: code = NotFound desc = could not find container \"b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c\": container with ID starting with b2fa3a4025c3b0de681d9ac377e5f8af995630c63ab2ce5531a6c4eae84b2e4c not found: ID does not exist"
Oct 30 21:26:14.415161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.415120    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"]
Oct 30 21:26:14.421122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:14.421091    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2"]
Oct 30 21:26:15.337382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.336005    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:26:15.352242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.352195    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" path="/var/lib/kubelet/pods/0183f5a1-7bd4-4190-acb0-8dbe84649986/volumes"
Oct 30 21:26:15.372007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.371959    4549 generic.go:356] "Generic (PLEG): container finished" podID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerID="fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf" exitCode=0
Oct 30 21:26:15.372476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.372021    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"
Oct 30 21:26:15.372476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.372034    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" event={"ID":"dd146ff3-1f30-4048-9d66-f1eb69c45132","Type":"ContainerDied","Data":"fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf"}
Oct 30 21:26:15.372476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.372123    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq" event={"ID":"dd146ff3-1f30-4048-9d66-f1eb69c45132","Type":"ContainerDied","Data":"d96f521ef2449eb2687db920b998564e5df642fcb574adf7e34fd896fdc4c094"}
Oct 30 21:26:15.372476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.372183    4549 scope.go:122] "RemoveContainer" containerID="fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf"
Oct 30 21:26:15.394111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.394087    4549 scope.go:122] "RemoveContainer" containerID="0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"
Oct 30 21:26:15.408056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.408031    4549 scope.go:122] "RemoveContainer" containerID="fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf"
Oct 30 21:26:15.408492 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:15.408455    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf\": container with ID starting with fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf not found: ID does not exist" containerID="fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf"
Oct 30 21:26:15.408615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.408523    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf"} err="failed to get container status \"fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf\": rpc error: code = NotFound desc = could not find container \"fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf\": container with ID starting with fb3a89f01621a689ed1b697e68b360c612342bda90ead66d78844420aee52dcf not found: ID does not exist"
Oct 30 21:26:15.408615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.408548    4549 scope.go:122] "RemoveContainer" containerID="0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"
Oct 30 21:26:15.408924 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:15.408899    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0\": container with ID starting with 0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0 not found: ID does not exist" containerID="0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"
Oct 30 21:26:15.409003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.408934    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0"} err="failed to get container status \"0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0\": rpc error: code = NotFound desc = could not find container \"0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0\": container with ID starting with 0864ffc078cc1c68f1cd634646691ad3ed0afcaf3d8c744e2139b15a2edbfbf0 not found: ID does not exist"
Oct 30 21:26:15.416270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416240    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.416409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416301    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.416502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416465    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.416570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416538    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.416626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416604    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.416683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416672    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.416920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416880    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.417004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.416924    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.417046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.417017    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs\") pod \"dd146ff3-1f30-4048-9d66-f1eb69c45132\" (UID: \"dd146ff3-1f30-4048-9d66-f1eb69c45132\") "
Oct 30 21:26:15.417081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.417037    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.417970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.417946    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.419231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.419201    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.419337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.419278    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.419419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.419392    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "kube-api-access-cg8c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:15.419637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.419557    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.419727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.419669    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.419862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.419819    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:15.420973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.420946    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:15.425359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.425326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.433064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.433038    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo" pod "dd146ff3-1f30-4048-9d66-f1eb69c45132" (UID: "dd146ff3-1f30-4048-9d66-f1eb69c45132"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519802    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd146ff3-1f30-4048-9d66-f1eb69c45132-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519861    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519876    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519891    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519904    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519920    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd146ff3-1f30-4048-9d66-f1eb69c45132-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519934    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg8c7\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-kube-api-access-cg8c7\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.519948    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd146ff3-1f30-4048-9d66-f1eb69c45132-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:15.586909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.586873    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.733306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.733266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.748942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.748908    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"]
Oct 30 21:26:15.751330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.751299    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq"]
Oct 30 21:26:16.001949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:15.987408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:16.185908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:16.185424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:17.347621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.347586    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" path="/var/lib/kubelet/pods/dd146ff3-1f30-4048-9d66-f1eb69c45132/volumes"
Oct 30 21:26:17.493248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.493212    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:17.643344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.643257    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"]
Oct 30 21:26:17.644091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644075    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy"
Oct 30 21:26:17.644137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644092    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy"
Oct 30 21:26:17.644137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644105    4549 cpu_manager.go:414] "containerMap: removing container" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy"
Oct 30 21:26:17.644137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644110    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy"
Oct 30 21:26:17.644137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644128    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy"
Oct 30 21:26:17.644137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644134    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy"
Oct 30 21:26:17.644300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644246    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy"
Oct 30 21:26:17.644300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644255    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy"
Oct 30 21:26:17.644300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644265    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy"
Oct 30 21:26:17.644300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.644271    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0183f5a1-7bd4-4190-acb0-8dbe84649986" containerName="istio-proxy"
Oct 30 21:26:17.647101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.647082    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:17.654541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.654512    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:17.672868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.672781    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"]
Oct 30 21:26:17.678730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.678699    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c5145c-dockercfg-mpvdm\""
Oct 30 21:26:17.678730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.678720    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c5145c\""
Oct 30 21:26:17.691032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.690994    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"]
Oct 30 21:26:17.692372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.692348    4549 cpu_manager.go:414] "containerMap: removing container" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy"
Oct 30 21:26:17.692429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.692376    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd146ff3-1f30-4048-9d66-f1eb69c45132" containerName="istio-proxy"
Oct 30 21:26:17.696244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.696220    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:17.724455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.724422    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"]
Oct 30 21:26:17.784550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.784510    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbm5s\" (UniqueName: \"kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s\") pod \"payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:17.785120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.785090    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4m4h\" (UniqueName: \"kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h\") pod \"payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:17.800163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.800049    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:17.801222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.801187    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:17.916066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.915954    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4m4h\" (UniqueName: \"kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h\") pod \"payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:17.916568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.916538    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:17.916816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.916790    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:17.917127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.917106    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbm5s\" (UniqueName: \"kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s\") pod \"payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:17.918851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.918804    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:17.928869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:17.919722    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:18.058240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:18.053385    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbm5s\" (UniqueName: \"kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s\") pod \"payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:18.058240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:18.055556    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4m4h\" (UniqueName: \"kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h\") pod \"payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:18.257897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:18.257773    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:18.262918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:18.262889    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:18.317348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:18.317319    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:18.779464 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:18.779429    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87577dca_ef17_455d_80b2_8dcf5728f362.slice/crio-690d4a599b3b1d11a13d2a9399e13e64b6d6eb9d3259c3b8468194d40f7630cd WatchSource:0}: Error finding container 690d4a599b3b1d11a13d2a9399e13e64b6d6eb9d3259c3b8468194d40f7630cd: Status 404 returned error can't find the container with id 690d4a599b3b1d11a13d2a9399e13e64b6d6eb9d3259c3b8468194d40f7630cd
Oct 30 21:26:18.781700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:18.781670    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"]
Oct 30 21:26:19.011851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.010689    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"]
Oct 30 21:26:19.600556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.588989    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" event={"ID":"4407ac7e-75a7-4990-9585-8d4f9502e5e0","Type":"ContainerStarted","Data":"ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740"}
Oct 30 21:26:19.600556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.589058    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" event={"ID":"4407ac7e-75a7-4990-9585-8d4f9502e5e0","Type":"ContainerStarted","Data":"51ef1ff7c4bb22ebfe4c63288d526f55788cffdc521a5b75ac933237a4584c27"}
Oct 30 21:26:19.600556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.598211    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:19.604853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.603278    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5" event={"ID":"87577dca-ef17-455d-80b2-8dcf5728f362","Type":"ContainerStarted","Data":"09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13"}
Oct 30 21:26:19.604853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.603343    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5" event={"ID":"87577dca-ef17-455d-80b2-8dcf5728f362","Type":"ContainerStarted","Data":"690d4a599b3b1d11a13d2a9399e13e64b6d6eb9d3259c3b8468194d40f7630cd"}
Oct 30 21:26:19.604853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.604607    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:19.678995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.678937    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" podStartSLOduration=2.678920535 podStartE2EDuration="2.678920535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:19.668304706 +0000 UTC m=+1562.442704646" watchObservedRunningTime="2026-10-30 21:26:19.678920535 +0000 UTC m=+1562.453320474"
Oct 30 21:26:19.766696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.766640    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5" podStartSLOduration=2.766626548 podStartE2EDuration="2.766626548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:19.763693631 +0000 UTC m=+1562.538093570" watchObservedRunningTime="2026-10-30 21:26:19.766626548 +0000 UTC m=+1562.541026516"
Oct 30 21:26:19.913357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:19.913230    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:20.694805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:20.694761    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:21.397796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:21.397764    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:23.006791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:23.006756    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:24.399468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:24.399426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:26.199471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:26.199431    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:26.576617 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:26.576574    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:26:26.734675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:26.734641    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:26.873079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:26.872994    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:27.025394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:27.025353    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"]
Oct 30 21:26:27.025643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:27.025609    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" containerID="cri-o://a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071" gracePeriod=30
Oct 30 21:26:27.095912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:27.095820    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:27.319909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:27.300690    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:27.817283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:27.817247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.197441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.197361    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"]
Oct 30 21:26:28.207632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.204628    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.238327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.238289    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"]
Oct 30 21:26:28.247681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.247653    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-290ede-openshift-default-dockercfg-z485j\""
Oct 30 21:26:28.258932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.258897    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mxv2\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259033    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259107    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259138    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259173    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259284    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259369    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259475    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.259705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.259686    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.391857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.391204    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.391857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.391769    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mxv2\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.391866    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.391928    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.391952    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.391989    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.392040    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.392103    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.392446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.392185    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.393317    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.398686    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.404223    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.404512    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.405020    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.405734    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.438905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.420198    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.587851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.583390    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.659852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.651858    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mxv2\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.726216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.725953    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.727490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.727437    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token\") pod \"e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") " pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.841063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.840979    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:28.874869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:28.873195    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:29.229512 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:29.229484    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5540263_fc2a_4fc6_b9c4_7045118690ee.slice/crio-9ed066e6648b345cc8a18273dc3469d06a60a65d843f52ff73ea728fce55e706 WatchSource:0}: Error finding container 9ed066e6648b345cc8a18273dc3469d06a60a65d843f52ff73ea728fce55e706: Status 404 returned error can't find the container with id 9ed066e6648b345cc8a18273dc3469d06a60a65d843f52ff73ea728fce55e706
Oct 30 21:26:29.230980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.230951    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"]
Oct 30 21:26:29.232627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.232595    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:29.232697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.232663    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:29.232753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.232695    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:29.369708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.369673    4549 patch_prober.go:29] interesting pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:26:29.369944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.369744    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:26:29.738788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.738751    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" event={"ID":"a5540263-fc2a-4fc6-b9c4-7045118690ee","Type":"ContainerStarted","Data":"90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"}
Oct 30 21:26:29.739328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.738807    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" event={"ID":"a5540263-fc2a-4fc6-b9c4-7045118690ee","Type":"ContainerStarted","Data":"9ed066e6648b345cc8a18273dc3469d06a60a65d843f52ff73ea728fce55e706"}
Oct 30 21:26:29.792333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.792284    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podStartSLOduration=1.792268005 podStartE2EDuration="1.792268005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:29.791336771 +0000 UTC m=+1572.565736714" watchObservedRunningTime="2026-10-30 21:26:29.792268005 +0000 UTC m=+1572.566667943"
Oct 30 21:26:29.841173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:29.841132    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:30.328667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:30.328635    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:30.615226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:30.615133    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:26:30.847556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:30.847530    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:31.080118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:31.080085    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:31.621899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:31.621866    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:26:31.754418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:31.754376    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:31.755584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:31.755540    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:26:32.235687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.233760    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.456881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.456802    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.489597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.489567    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:26:32.652530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652488    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652553    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652594    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652712    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652806    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652906    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652953    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.652996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.652945    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:32.653212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.653004    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.653212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.653044    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:32.653212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.653116    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs\") pod \"b44e1c2a-6eef-44fc-817e-83b88588f91b\" (UID: \"b44e1c2a-6eef-44fc-817e-83b88588f91b\") "
Oct 30 21:26:32.653364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.653230    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:32.653590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.653496    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:32.653731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.653706    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:32.654340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.654313    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.654519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.654343    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.654519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.654359    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.654519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.654375    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b44e1c2a-6eef-44fc-817e-83b88588f91b-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.654519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.654388    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.655291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.655249    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:32.655396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.655367    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:32.655505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.655480    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "kube-api-access-qsl67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:32.655728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.655709    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token" pod "b44e1c2a-6eef-44fc-817e-83b88588f91b" (UID: "b44e1c2a-6eef-44fc-817e-83b88588f91b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:32.756404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.756321    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.756404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.756359    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.756404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.756374    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsl67\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-kube-api-access-qsl67\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.756404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.756388    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b44e1c2a-6eef-44fc-817e-83b88588f91b-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:32.764190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.764157    4549 generic.go:356] "Generic (PLEG): container finished" podID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerID="a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071" exitCode=0
Oct 30 21:26:32.764359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.764206    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"
Oct 30 21:26:32.764359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.764237    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" event={"ID":"b44e1c2a-6eef-44fc-817e-83b88588f91b","Type":"ContainerDied","Data":"a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071"}
Oct 30 21:26:32.764359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.764302    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl" event={"ID":"b44e1c2a-6eef-44fc-817e-83b88588f91b","Type":"ContainerDied","Data":"ea52d112dd29f6108b26cc6eee24629f6a598faaf43e0bb894be07bcc9308637"}
Oct 30 21:26:32.764359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.764326    4549 scope.go:122] "RemoveContainer" containerID="a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071"
Oct 30 21:26:32.777643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.777620    4549 scope.go:122] "RemoveContainer" containerID="8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"
Oct 30 21:26:32.788349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.788327    4549 scope.go:122] "RemoveContainer" containerID="a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071"
Oct 30 21:26:32.788709 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:32.788684    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071\": container with ID starting with a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071 not found: ID does not exist" containerID="a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071"
Oct 30 21:26:32.788794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.788719    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071"} err="failed to get container status \"a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071\": rpc error: code = NotFound desc = could not find container \"a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071\": container with ID starting with a5cb0dd1a17b012f339d1b1e8a889ec7fc10bd7b948ba34880ea11bf14bc7071 not found: ID does not exist"
Oct 30 21:26:32.788794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.788739    4549 scope.go:122] "RemoveContainer" containerID="8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"
Oct 30 21:26:32.789028 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:32.789011    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149\": container with ID starting with 8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149 not found: ID does not exist" containerID="8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"
Oct 30 21:26:32.789091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.789031    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149"} err="failed to get container status \"8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149\": rpc error: code = NotFound desc = could not find container \"8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149\": container with ID starting with 8487c0e0059dfda56d669090428bab54b7c8f8796f15e12651033530ce102149 not found: ID does not exist"
Oct 30 21:26:32.820850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.820801    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"]
Oct 30 21:26:32.829682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:32.829642    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl"]
Oct 30 21:26:33.348407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:33.348373    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" path="/var/lib/kubelet/pods/b44e1c2a-6eef-44fc-817e-83b88588f91b/volumes"
Oct 30 21:26:33.604095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:33.604014    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"]
Oct 30 21:26:33.604297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:33.604262    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy" containerID="cri-o://11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82" gracePeriod=30
Oct 30 21:26:34.099174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.099132    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.236805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.236770    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"]
Oct 30 21:26:34.237639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.237622    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy"
Oct 30 21:26:34.237709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.237641    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy"
Oct 30 21:26:34.237709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.237682    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy"
Oct 30 21:26:34.237709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.237688    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy"
Oct 30 21:26:34.237808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.237793    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy"
Oct 30 21:26:34.237808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.237804    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b44e1c2a-6eef-44fc-817e-83b88588f91b" containerName="istio-proxy"
Oct 30 21:26:34.243691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.243672    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.263686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.263653    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-290ede-dockercfg-tp5fk\""
Oct 30 21:26:34.263937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.263906    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-290ede\""
Oct 30 21:26:34.264447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.264427    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"]
Oct 30 21:26:34.274051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.274016    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.281083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.281053    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.281263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.281102    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppgdr\" (UniqueName: \"kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr\") pod \"payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.530917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.530803    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.535934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.535904    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"]
Oct 30 21:26:34.773066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.731002    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.776594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.774076    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppgdr\" (UniqueName: \"kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr\") pod \"payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.782446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.782370    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.784913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.784651    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"]
Oct 30 21:26:34.784913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.784793    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:34.812482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.811147    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppgdr\" (UniqueName: \"kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr\") pod \"payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.858910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.858878    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:34.880011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.879981    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:34.880144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.880132    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lzcw\" (UniqueName: \"kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw\") pod \"payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:34.981485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.981448    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:34.981681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.981578    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lzcw\" (UniqueName: \"kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw\") pod \"payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:34.982382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:34.982357    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:35.004372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.004333    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lzcw\" (UniqueName: \"kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw\") pod \"payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:35.152328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.152291    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:35.174664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.174634    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"]
Oct 30 21:26:35.176021 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:35.175968    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537612db_c727_42c3_b8a7_febe5d6e1812.slice/crio-732980f7d12f7040524b747a60ff87eb8001b3079df9d118679ca674a2c0ad49 WatchSource:0}: Error finding container 732980f7d12f7040524b747a60ff87eb8001b3079df9d118679ca674a2c0ad49: Status 404 returned error can't find the container with id 732980f7d12f7040524b747a60ff87eb8001b3079df9d118679ca674a2c0ad49
Oct 30 21:26:35.193551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.193504    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:35.354193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.354161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:35.475190 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:35.475151    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b1a8bf_adc9_44a6_bcf0_510c985a735a.slice/crio-396d8d7a7762451e127410e35bb76dd0cdf90b3a96d4c86c4fed5d39d0bf72cc WatchSource:0}: Error finding container 396d8d7a7762451e127410e35bb76dd0cdf90b3a96d4c86c4fed5d39d0bf72cc: Status 404 returned error can't find the container with id 396d8d7a7762451e127410e35bb76dd0cdf90b3a96d4c86c4fed5d39d0bf72cc
Oct 30 21:26:35.475308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.475218    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"]
Oct 30 21:26:35.822627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.822571    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" event={"ID":"537612db-c727-42c3-b8a7-febe5d6e1812","Type":"ContainerStarted","Data":"843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219"}
Oct 30 21:26:35.823163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.822666    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:35.823163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.822692    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" event={"ID":"537612db-c727-42c3-b8a7-febe5d6e1812","Type":"ContainerStarted","Data":"732980f7d12f7040524b747a60ff87eb8001b3079df9d118679ca674a2c0ad49"}
Oct 30 21:26:35.824696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.824669    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" event={"ID":"53b1a8bf-adc9-44a6-bcf0-510c985a735a","Type":"ContainerStarted","Data":"e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be"}
Oct 30 21:26:35.824865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.824713    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" event={"ID":"53b1a8bf-adc9-44a6-bcf0-510c985a735a","Type":"ContainerStarted","Data":"396d8d7a7762451e127410e35bb76dd0cdf90b3a96d4c86c4fed5d39d0bf72cc"}
Oct 30 21:26:35.824865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.824849    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:35.865574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.865515    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" podStartSLOduration=1.865495838 podStartE2EDuration="1.865495838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:35.862706271 +0000 UTC m=+1578.637106210" watchObservedRunningTime="2026-10-30 21:26:35.865495838 +0000 UTC m=+1578.639895778"
Oct 30 21:26:35.907350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.907283    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" podStartSLOduration=1.907267369 podStartE2EDuration="1.907267369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:35.902211129 +0000 UTC m=+1578.676611068" watchObservedRunningTime="2026-10-30 21:26:35.907267369 +0000 UTC m=+1578.681667338"
Oct 30 21:26:35.935251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.935214    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"]
Oct 30 21:26:35.935568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:35.935479    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv" podUID="2c07aa3a-41bf-416e-999f-0e7e7a08fa37" containerName="maas-api" containerID="cri-o://018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a" gracePeriod=30
Oct 30 21:26:36.039172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.039134    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.693417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.693372    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:26:36.810695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.796676    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca\") pod \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") "
Oct 30 21:26:36.810695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.796790    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld\" (UniqueName: \"kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld\") pod \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") "
Oct 30 21:26:36.810695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.796917    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs\") pod \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") "
Oct 30 21:26:36.810695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.797043    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls\") pod \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\" (UID: \"2c07aa3a-41bf-416e-999f-0e7e7a08fa37\") "
Oct 30 21:26:36.811254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.811218    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca" pod "2c07aa3a-41bf-416e-999f-0e7e7a08fa37" (UID: "2c07aa3a-41bf-416e-999f-0e7e7a08fa37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:36.811504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.811474    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls" pod "2c07aa3a-41bf-416e-999f-0e7e7a08fa37" (UID: "2c07aa3a-41bf-416e-999f-0e7e7a08fa37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:26:36.883755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.846127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.883755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.855243    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld" pod "2c07aa3a-41bf-416e-999f-0e7e7a08fa37" (UID: "2c07aa3a-41bf-416e-999f-0e7e7a08fa37"). InnerVolumeSpecName "kube-api-access-rtgld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:36.887180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.886401    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs" pod "2c07aa3a-41bf-416e-999f-0e7e7a08fa37" (UID: "2c07aa3a-41bf-416e-999f-0e7e7a08fa37"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:26:36.951282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.950966    4549 generic.go:356] "Generic (PLEG): container finished" podID="2c07aa3a-41bf-416e-999f-0e7e7a08fa37" containerID="018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a" exitCode=0
Oct 30 21:26:37.022857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:36.994900    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"
Oct 30 21:26:37.076356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.074497    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.076356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.074921    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtgld\" (UniqueName: \"kubernetes.io/projected/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-kube-api-access-rtgld\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:37.076356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.074959    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:37.076356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.074984    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:37.076356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.074998    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c07aa3a-41bf-416e-999f-0e7e7a08fa37-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:37.076356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.010511    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv" event={"ID":"2c07aa3a-41bf-416e-999f-0e7e7a08fa37","Type":"ContainerDied","Data":"018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a"}
Oct 30 21:26:37.076910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.076393    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv" event={"ID":"2c07aa3a-41bf-416e-999f-0e7e7a08fa37","Type":"ContainerDied","Data":"1530054779d2e07c2006948b249ff389628de6f9af4aedec939fe30268965b52"}
Oct 30 21:26:37.076910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.076448    4549 scope.go:122] "RemoveContainer" containerID="018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a"
Oct 30 21:26:37.294772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.280963    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.300592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.300559    4549 scope.go:122] "RemoveContainer" containerID="018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a"
Oct 30 21:26:37.305144 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:37.304873    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a\": container with ID starting with 018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a not found: ID does not exist" containerID="018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a"
Oct 30 21:26:37.308012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.307180    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a"} err="failed to get container status \"018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a\": rpc error: code = NotFound desc = could not find container \"018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a\": container with ID starting with 018ad943b5556d1f3d36263e37d5b0bef2313b43b7a5359e7aeb6ae73f4bdf9a not found: ID does not exist"
Oct 30 21:26:37.353554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.353527    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"]
Oct 30 21:26:37.364432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.364395    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ca18b3-6b4b8585d7-zswzv"]
Oct 30 21:26:37.631684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:37.631655    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:38.363890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.363067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:38.898298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.898268    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:26:38.919541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.919510    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:38.989872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.989677    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.989872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.989778    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.989893    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.989946    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.990032    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.990062    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.990255    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.990398    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.990483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.990453    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert\") pod \"08643223-5f4c-44cd-be86-9c9c7aee0a84\" (UID: \"08643223-5f4c-44cd-be86-9c9c7aee0a84\") "
Oct 30 21:26:38.995585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.992473    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:38.995585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.993295    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:38.995839 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.995678    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:38.997223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.997191    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:39.000365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:38.999940    4549 generic.go:356] "Generic (PLEG): container finished" podID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerID="11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82" exitCode=0
Oct 30 21:26:39.000365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.000192    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" event={"ID":"08643223-5f4c-44cd-be86-9c9c7aee0a84","Type":"ContainerDied","Data":"11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82"}
Oct 30 21:26:39.000365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.000244    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2" event={"ID":"08643223-5f4c-44cd-be86-9c9c7aee0a84","Type":"ContainerDied","Data":"817e41d128027e22bcdb23b6c93644b2479454aaf57f109c2240f641fb3cac29"}
Oct 30 21:26:39.000365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.000273    4549 scope.go:122] "RemoveContainer" containerID="11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82"
Oct 30 21:26:39.000365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.000305    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"
Oct 30 21:26:39.000702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.000364    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:39.001059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.001035    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:39.001919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.001810    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:39.006338 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.006312    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "kube-api-access-2vws7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:39.009861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.009101    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo" pod "08643223-5f4c-44cd-be86-9c9c7aee0a84" (UID: "08643223-5f4c-44cd-be86-9c9c7aee0a84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096722    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096764    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096781    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08643223-5f4c-44cd-be86-9c9c7aee0a84-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096795    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096810    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096823    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08643223-5f4c-44cd-be86-9c9c7aee0a84-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096857    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vws7\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-kube-api-access-2vws7\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096871    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.096886    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08643223-5f4c-44cd-be86-9c9c7aee0a84-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.101103    4549 scope.go:122] "RemoveContainer" containerID="11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82"
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:39.101590    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82\": container with ID starting with 11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82 not found: ID does not exist" containerID="11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82"
Oct 30 21:26:39.103903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.101629    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82"} err="failed to get container status \"11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82\": rpc error: code = NotFound desc = could not find container \"11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82\": container with ID starting with 11bb46dcf04b0e53cfea118031a2f0b7451c8095cfca51163d19d54addac1f82 not found: ID does not exist"
Oct 30 21:26:39.347790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.347752    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c07aa3a-41bf-416e-999f-0e7e7a08fa37" path="/var/lib/kubelet/pods/2c07aa3a-41bf-416e-999f-0e7e7a08fa37/volumes"
Oct 30 21:26:39.349317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.349293    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"]
Oct 30 21:26:39.359704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:39.359675    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2"]
Oct 30 21:26:40.648855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:40.639612    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:41.382299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:41.382259    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" path="/var/lib/kubelet/pods/08643223-5f4c-44cd-be86-9c9c7aee0a84/volumes"
Oct 30 21:26:43.023724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:43.023682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:44.622899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:44.622546    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:44.870163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:44.870120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.759452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.759412    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.765161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.763419    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"]
Oct 30 21:26:45.766621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.766596    4549 cpu_manager.go:414] "containerMap: removing container" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy"
Oct 30 21:26:45.766768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.766753    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy"
Oct 30 21:26:45.766966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.766954    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2c07aa3a-41bf-416e-999f-0e7e7a08fa37" containerName="maas-api"
Oct 30 21:26:45.767066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.767054    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c07aa3a-41bf-416e-999f-0e7e7a08fa37" containerName="maas-api"
Oct 30 21:26:45.767339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.767325    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c07aa3a-41bf-416e-999f-0e7e7a08fa37" containerName="maas-api"
Oct 30 21:26:45.767453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.767443    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="08643223-5f4c-44cd-be86-9c9c7aee0a84" containerName="istio-proxy"
Oct 30 21:26:45.828774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.828739    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.837308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.837279    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"]
Oct 30 21:26:45.857058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.857033    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b2628a-openshift-default-dockercfg-rdbgl\""
Oct 30 21:26:45.865920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.865885    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.865924    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.865944    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.866012    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgxjq\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.866134    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.866288    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.866527    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.866594    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.866634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.866628    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.946294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.946257    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.997817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.997783    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.998850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.998808    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:45.999376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.999358    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.026283    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.998471    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.026172    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.027414    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:45.999257    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.027669    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.027746    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.027803    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgxjq\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.027880    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.028379    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.030898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.028945    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.057865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.057524    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.064858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.063220    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.064858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.064509    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgxjq\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.065104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.065042    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs\") pod \"e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") " pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.148964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.148928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.156314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.156276    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:46.310695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.310532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.675862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.674408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.801250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.800308    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"]
Oct 30 21:26:46.801250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:46.800554    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy" containerID="cri-o://fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1" gracePeriod=30
Oct 30 21:26:47.079966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.065503    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"]
Oct 30 21:26:47.079966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.066562    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" containerID="cri-o://81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33" gracePeriod=30
Oct 30 21:26:47.079966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.073346    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:26:47.079966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.076091    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:26:47.209854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.178292    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"]
Oct 30 21:26:47.209854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.178615    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy" containerID="cri-o://3da33c59cad0832266c3b6f0302a19ae3be0eeaeb575689048b06caa75bdb2f6" gracePeriod=30
Oct 30 21:26:47.249034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.248957    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"]
Oct 30 21:26:47.268230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.268189    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:47.268325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.268274    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:47.268325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.268314    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:47.449617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.449584    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" event={"ID":"a88d883a-bb1c-4c36-8a95-6911ac25325d","Type":"ContainerStarted","Data":"a2471cc72fd109ccd31f381c45453b8e29dc0e598e34d2b8160efa440c3d097e"}
Oct 30 21:26:47.721124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:47.716408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:48.459711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.459674    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" event={"ID":"a88d883a-bb1c-4c36-8a95-6911ac25325d","Type":"ContainerStarted","Data":"49ca274d2befbb500fc205c781f0e3ebad381a79abf5c4adbb83ed2f41067b3e"}
Oct 30 21:26:48.534480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.534429    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podStartSLOduration=3.5344127260000002 podStartE2EDuration="3.534412726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:48.52330818 +0000 UTC m=+1591.297708120" watchObservedRunningTime="2026-10-30 21:26:48.534412726 +0000 UTC m=+1591.308812730"
Oct 30 21:26:48.732523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.732425    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"]
Oct 30 21:26:48.743153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.743127    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.752490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.752449    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"]
Oct 30 21:26:48.759707 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.759684    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c31999\"/\"default-dockercfg-2r4qr\""
Oct 30 21:26:48.759877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.759682    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c31999\"/\"openshift-service-ca.crt\""
Oct 30 21:26:48.759962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.759684    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c31999\"/\"kube-root-ca.crt\""
Oct 30 21:26:48.760002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.759869    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c31999\"/\"test-model-c31999-kserve-self-signed-certs\""
Oct 30 21:26:48.831399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.831351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:48.832165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.832134    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.832302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.832186    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qh4f\" (UniqueName: \"kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.832481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.832460    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.832701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.832674    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.832853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.832717    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.832931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.832814    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.934749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.934716    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.934988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.934763    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qh4f\" (UniqueName: \"kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.934988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.934850    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.935131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.935020    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.935131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.935060    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.935131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.935112    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.935389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.935371    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.935524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.935502    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.935929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.935601    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.937310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.937290    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.937812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.937793    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:48.967868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:48.955124    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qh4f\" (UniqueName: \"kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f\") pod \"test-model-c31999-kserve-5748d95599-x7s6p\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") " pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:49.081900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.066278    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:49.172550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.168151    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:49.236860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.210878    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:49.536437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.536042    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:49.539581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.539555    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:26:49.910745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.910711    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"]
Oct 30 21:26:49.910916 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:49.910889    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bbb76b0_9f3a_4686_b111_1c7d2f27a55a.slice/crio-1fc69c37a6430f0ca48bf28ef09bfadcb7d5ddc45d9fac45d4fac5650a002baf WatchSource:0}: Error finding container 1fc69c37a6430f0ca48bf28ef09bfadcb7d5ddc45d9fac45d4fac5650a002baf: Status 404 returned error can't find the container with id 1fc69c37a6430f0ca48bf28ef09bfadcb7d5ddc45d9fac45d4fac5650a002baf
Oct 30 21:26:49.924209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:49.924085    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:50.544895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:50.544863    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" event={"ID":"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a","Type":"ContainerStarted","Data":"fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095"}
Oct 30 21:26:50.545295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:50.544906    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" event={"ID":"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a","Type":"ContainerStarted","Data":"1fc69c37a6430f0ca48bf28ef09bfadcb7d5ddc45d9fac45d4fac5650a002baf"}
Oct 30 21:26:50.601794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:50.601728    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" podStartSLOduration=2.601706914 podStartE2EDuration="2.601706914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:50.596883655 +0000 UTC m=+1593.371283595" watchObservedRunningTime="2026-10-30 21:26:50.601706914 +0000 UTC m=+1593.376106855"
Oct 30 21:26:51.033470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:51.033372    4549 patch_prober.go:29] interesting pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:26:51.033470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:51.033456    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:26:51.066705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:51.066672    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:51.468503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:51.468464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:51.631207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:51.631167    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:51.825855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:51.825310    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:52.405119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.402012    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:26:52.496971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.496939    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:26:52.538582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.538540    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.538981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.538865    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.539101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.539032    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.539547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.539515    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.539690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.539672    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.540174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.540150    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.541323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.541299    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.541602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.541576    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.541742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.541724    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.541997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.541971    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.542123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.542106    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.542543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.542515    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:52.542712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.542691    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.543107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.543085    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96\") pod \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\" (UID: \"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7\") "
Oct 30 21:26:52.544309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.544282    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.544404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.544313    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.544404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.544329    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.544404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.544343    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.544404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.544355    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.546522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.546495    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.547450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.547421    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:52.549532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.549502    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:52.550122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.550083    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96" pod "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" (UID: "85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7"). InnerVolumeSpecName "kube-api-access-wnt96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:52.586433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.586367    4549 generic.go:356] "Generic (PLEG): container finished" podID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerID="fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1" exitCode=0
Oct 30 21:26:52.586671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.586562    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" event={"ID":"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7","Type":"ContainerDied","Data":"fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1"}
Oct 30 21:26:52.586671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.586617    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk" event={"ID":"85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7","Type":"ContainerDied","Data":"35a5e5155ded2c25d85cda5752c9d0f75bd7c216c660acaf582d9a4086d0d5eb"}
Oct 30 21:26:52.586671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.586647    4549 scope.go:122] "RemoveContainer" containerID="fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1"
Oct 30 21:26:52.586946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.586844    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"
Oct 30 21:26:52.601797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.601764    4549 generic.go:356] "Generic (PLEG): container finished" podID="893d4eeb-8daf-4865-b633-de3eed97661e" containerID="81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33" exitCode=0
Oct 30 21:26:52.602499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.602077    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"
Oct 30 21:26:52.605426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.602097    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" event={"ID":"893d4eeb-8daf-4865-b633-de3eed97661e","Type":"ContainerDied","Data":"81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33"}
Oct 30 21:26:52.605621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.605602    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c" event={"ID":"893d4eeb-8daf-4865-b633-de3eed97661e","Type":"ContainerDied","Data":"22e0ff1fbebb78be1052bbbbabb416953e5595e0320fb05bd3f2e39e884a2fb4"}
Oct 30 21:26:52.605754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.605742    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" event={"ID":"74e9a27f-b860-4a10-a145-2513437578ad","Type":"ContainerDied","Data":"3da33c59cad0832266c3b6f0302a19ae3be0eeaeb575689048b06caa75bdb2f6"}
Oct 30 21:26:52.605935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.605157    4549 generic.go:356] "Generic (PLEG): container finished" podID="74e9a27f-b860-4a10-a145-2513437578ad" containerID="3da33c59cad0832266c3b6f0302a19ae3be0eeaeb575689048b06caa75bdb2f6" exitCode=0
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.614644    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"]
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.615766    4549 cpu_manager.go:414] "containerMap: removing container" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.615785    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.615818    4549 cpu_manager.go:414] "containerMap: removing container" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.615844    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.615868    4549 cpu_manager.go:414] "containerMap: removing container" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.615875    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.616099    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.616117    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" containerName="istio-proxy"
Oct 30 21:26:52.618401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.616131    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" containerName="istio-proxy"
Oct 30 21:26:52.622519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.621886    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.641473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.640179    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"]
Oct 30 21:26:52.645245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.645216    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.645781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.645755    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.645995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.645974    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.646237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646220    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.646440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646423    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.646582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646566    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.646706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646694    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.646857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646099    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.647064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646593    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.647161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646811    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.647238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.647069    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.647315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.646886    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.647450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.647434    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token\") pod \"893d4eeb-8daf-4865-b633-de3eed97661e\" (UID: \"893d4eeb-8daf-4865-b633-de3eed97661e\") "
Oct 30 21:26:52.647726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.647706    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:52.648628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.648606    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.648763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.648749    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.648882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.648869    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnt96\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-kube-api-access-wnt96\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.648990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.648976    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.649081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.649070    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.649169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.649159    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.649249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.649240    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.649335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.649324    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.649418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.649408    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893d4eeb-8daf-4865-b633-de3eed97661e-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.649630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.649598    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "kube-api-access-bxlj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:52.650341 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.650306    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.651974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.651947    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:52.653162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.653142    4549 scope.go:122] "RemoveContainer" containerID="fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1"
Oct 30 21:26:52.653534 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:52.653507    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1\": container with ID starting with fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1 not found: ID does not exist" containerID="fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1"
Oct 30 21:26:52.653639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.653541    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1"} err="failed to get container status \"fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1\": rpc error: code = NotFound desc = could not find container \"fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1\": container with ID starting with fbd1542b1e482fba30bd053aa99c2cb1ef3e52618b8579f5bc7be9bb01be33f1 not found: ID does not exist"
Oct 30 21:26:52.653639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.653566    4549 scope.go:122] "RemoveContainer" containerID="81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33"
Oct 30 21:26:52.655383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.655355    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token" pod "893d4eeb-8daf-4865-b633-de3eed97661e" (UID: "893d4eeb-8daf-4865-b633-de3eed97661e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:52.669170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.669073    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:26:52.684018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.683966    4549 scope.go:122] "RemoveContainer" containerID="c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"
Oct 30 21:26:52.697285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.697205    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1eab65-openshift-default-dockercfg-5jsjw\""
Oct 30 21:26:52.705288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.704518    4549 scope.go:122] "RemoveContainer" containerID="81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33"
Oct 30 21:26:52.705288 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:52.704997    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33\": container with ID starting with 81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33 not found: ID does not exist" containerID="81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33"
Oct 30 21:26:52.705288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.705031    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33"} err="failed to get container status \"81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33\": rpc error: code = NotFound desc = could not find container \"81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33\": container with ID starting with 81b8668572f02be1cf437a384dd0c6eb8c87a9e2f89f5eee15f35aa0b4e6ef33 not found: ID does not exist"
Oct 30 21:26:52.705288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.705056    4549 scope.go:122] "RemoveContainer" containerID="c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"
Oct 30 21:26:52.705770 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:52.705677    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1\": container with ID starting with c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1 not found: ID does not exist" containerID="c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"
Oct 30 21:26:52.705770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.705708    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1"} err="failed to get container status \"c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1\": rpc error: code = NotFound desc = could not find container \"c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1\": container with ID starting with c87a966e9446d68d40cdbb196741ede494fd93ab82e68f720a5ab0b0d26c2bc1 not found: ID does not exist"
Oct 30 21:26:52.735861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.735808    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"]
Oct 30 21:26:52.743319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.743279    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk"]
Oct 30 21:26:52.750392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750355    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750414    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750492    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750585    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750619    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750749    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750845    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:52.750955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750882    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.750955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750894    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.750955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.750943    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.751116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751027    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo\") pod \"74e9a27f-b860-4a10-a145-2513437578ad\" (UID: \"74e9a27f-b860-4a10-a145-2513437578ad\") "
Oct 30 21:26:52.751116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751068    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.751290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751198    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.751571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751377    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751408    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751451    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751499    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751570    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751571    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.751938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751739    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751791    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.751938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.751858    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.752137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752064    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj5rv\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.752309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752292    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752315    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752329    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893d4eeb-8daf-4865-b633-de3eed97661e-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752344    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893d4eeb-8daf-4865-b633-de3eed97661e-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752359    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752372    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752390    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxlj7\" (UniqueName: \"kubernetes.io/projected/893d4eeb-8daf-4865-b633-de3eed97661e-kube-api-access-bxlj7\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752404    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74e9a27f-b860-4a10-a145-2513437578ad-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.752577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.752418    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.753733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.753696    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:52.754254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.754232    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:52.754433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.754410    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "kube-api-access-z7dbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:52.755248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.755222    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy" pod "74e9a27f-b860-4a10-a145-2513437578ad" (UID: "74e9a27f-b860-4a10-a145-2513437578ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:52.853885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.853806    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.854775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.854732    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.855678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855654    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.855884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855710    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.855884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855778    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.855884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855815    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.855884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855874    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.856272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855916    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.856272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.856091    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj5rv\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.856812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.856318    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74e9a27f-b860-4a10-a145-2513437578ad-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.856812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.856651    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.857269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.857232    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.857514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.857449    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.857514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.855578    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.857514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.857486    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.857737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.857543    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.860086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.857561    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74e9a27f-b860-4a10-a145-2513437578ad-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.860220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.860111    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7dbk\" (UniqueName: \"kubernetes.io/projected/74e9a27f-b860-4a10-a145-2513437578ad-kube-api-access-z7dbk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:26:52.860220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.860050    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.860425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.860395    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.885277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.885245    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj5rv\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.885442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.885304    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token\") pod \"e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") " pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.947302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.947268    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:52.962390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.962359    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"]
Oct 30 21:26:52.974191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:52.974152    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c"]
Oct 30 21:26:53.323914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.323724    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.364194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.364153    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7" path="/var/lib/kubelet/pods/85bf5ba2-23d2-4d01-a9eb-31a8c4c71be7/volumes"
Oct 30 21:26:53.365013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.364988    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="893d4eeb-8daf-4865-b633-de3eed97661e" path="/var/lib/kubelet/pods/893d4eeb-8daf-4865-b633-de3eed97661e/volumes"
Oct 30 21:26:53.365793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.365762    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"]
Oct 30 21:26:53.365909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.365843    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"]
Oct 30 21:26:53.374292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.374267    4549 cpu_manager.go:414] "containerMap: removing container" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy"
Oct 30 21:26:53.374471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.374454    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy"
Oct 30 21:26:53.374919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.374900    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="74e9a27f-b860-4a10-a145-2513437578ad" containerName="istio-proxy"
Oct 30 21:26:53.379891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.379710    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:53.379891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.379805    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:53.380302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.380118    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:53.392043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.390997    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"]
Oct 30 21:26:53.392043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.391147    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.440386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.440341    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"]
Oct 30 21:26:53.447954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.447923    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:53.472203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.472171    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"]
Oct 30 21:26:53.474169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.474149    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b2628a\""
Oct 30 21:26:53.483941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.483312    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.483941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.483385    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfv8t\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.483941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.483765    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.483941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.483822    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.484270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.484110    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.484327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.484299    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.484390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.484369    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.484627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.484578    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.484717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.484625    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.529309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.529246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.557029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.556529    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"]
Oct 30 21:26:53.564508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.564478    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.756176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.670369    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b2628a-dockercfg-fcfmz\""
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.843779    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwhzb\" (UniqueName: \"kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb\") pod \"payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.843868    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.843975    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844095    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844151    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844250    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844282    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844312    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844377    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844454    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844491    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844554    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844596    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfv8t\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.845200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.844696    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsstz\" (UniqueName: \"kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz\") pod \"payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.846078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.845480    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.846207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.846108    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.846380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.846356    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.861739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.861221    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.880516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.880478    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.937701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.935722    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.945187    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"]
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.946244    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.946402    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsstz\" (UniqueName: \"kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz\") pod \"payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.946439    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwhzb\" (UniqueName: \"kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb\") pod \"payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.946540    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.947641    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:53.948904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.948431    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.975695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.975659    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.989803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.989240    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfv8t\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t\") pod \"e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") " pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:53.992086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.991257    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsstz\" (UniqueName: \"kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz\") pod \"payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:53.995743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:53.995711    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwhzb\" (UniqueName: \"kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb\") pod \"payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:54.027228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.026875    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:54.084933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.083516    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" event={"ID":"16e9ef29-3164-4072-9560-251a9db10da5","Type":"ContainerStarted","Data":"f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12"}
Oct 30 21:26:54.084933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.083589    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" event={"ID":"16e9ef29-3164-4072-9560-251a9db10da5","Type":"ContainerStarted","Data":"049aa7c4e65b1d6fed00af352fc779931112d5e0de42d034cc680692b6d927f6"}
Oct 30 21:26:54.091726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.090523    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2" event={"ID":"74e9a27f-b860-4a10-a145-2513437578ad","Type":"ContainerDied","Data":"22a973273abcc70e54acd6843eac8f2122222b723119aebda78b252f94fcfa24"}
Oct 30 21:26:54.091726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.090601    4549 scope.go:122] "RemoveContainer" containerID="3da33c59cad0832266c3b6f0302a19ae3be0eeaeb575689048b06caa75bdb2f6"
Oct 30 21:26:54.091726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.090790    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"
Oct 30 21:26:54.107099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.107002    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:54.142212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.140017    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podStartSLOduration=2.139997244 podStartE2EDuration="2.139997244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:54.138979265 +0000 UTC m=+1596.913379206" watchObservedRunningTime="2026-10-30 21:26:54.139997244 +0000 UTC m=+1596.914397190"
Oct 30 21:26:54.205970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.205569    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:54.226176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.226057    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"]
Oct 30 21:26:54.243960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.243081    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2"]
Oct 30 21:26:54.330270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.330177    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:54.561443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.560992    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"]
Oct 30 21:26:54.567365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.565755    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:54.567365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.565900    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:54.567365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.565953    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:54.674891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.674822    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"]
Oct 30 21:26:54.680726 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:54.680698    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205e1b4d_a2c8_401c_bb17_a2af41b3f6b7.slice/crio-6c71cb50700cc985e830dafc23fff1eb638974f520f61fdb0e95b52e324e77ee WatchSource:0}: Error finding container 6c71cb50700cc985e830dafc23fff1eb638974f520f61fdb0e95b52e324e77ee: Status 404 returned error can't find the container with id 6c71cb50700cc985e830dafc23fff1eb638974f520f61fdb0e95b52e324e77ee
Oct 30 21:26:54.791513 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:26:54.791431    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde45254d_bdba_4f6f_83de_2fa8c41dc00d.slice/crio-d2accb37294bc0c90a397762332541da7b9f03b94a7dba4d4111a7f4b21b5ef8 WatchSource:0}: Error finding container d2accb37294bc0c90a397762332541da7b9f03b94a7dba4d4111a7f4b21b5ef8: Status 404 returned error can't find the container with id d2accb37294bc0c90a397762332541da7b9f03b94a7dba4d4111a7f4b21b5ef8
Oct 30 21:26:54.792045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.791619    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"]
Oct 30 21:26:54.947738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.947702    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:26:54.949551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.949520    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:26:54.949680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:54.949574    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:26:55.114840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.114704    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" event={"ID":"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7","Type":"ContainerStarted","Data":"3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6"}
Oct 30 21:26:55.114840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.114784    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" event={"ID":"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7","Type":"ContainerStarted","Data":"6c71cb50700cc985e830dafc23fff1eb638974f520f61fdb0e95b52e324e77ee"}
Oct 30 21:26:55.115192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.115150    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:26:55.119706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.119667    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" event={"ID":"de45254d-bdba-4f6f-83de-2fa8c41dc00d","Type":"ContainerStarted","Data":"fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc"}
Oct 30 21:26:55.120059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.120027    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" event={"ID":"de45254d-bdba-4f6f-83de-2fa8c41dc00d","Type":"ContainerStarted","Data":"d2accb37294bc0c90a397762332541da7b9f03b94a7dba4d4111a7f4b21b5ef8"}
Oct 30 21:26:55.121073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.121005    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:26:55.124023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.123988    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" event={"ID":"59e347c6-5c61-4579-93c3-f6a576f6cb42","Type":"ContainerStarted","Data":"01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee"}
Oct 30 21:26:55.124151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.124037    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" event={"ID":"59e347c6-5c61-4579-93c3-f6a576f6cb42","Type":"ContainerStarted","Data":"b0086d75fe7d84b1549971f7e40074ee6e60d99820aad0a64de6f5846aa025da"}
Oct 30 21:26:55.162729 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.162669    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" podStartSLOduration=2.1626492 podStartE2EDuration="2.1626492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:55.155342681 +0000 UTC m=+1597.929742621" watchObservedRunningTime="2026-10-30 21:26:55.1626492 +0000 UTC m=+1597.937049139"
Oct 30 21:26:55.198675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.198616    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" podStartSLOduration=2.198597425 podStartE2EDuration="2.198597425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:55.195808859 +0000 UTC m=+1597.970208800" watchObservedRunningTime="2026-10-30 21:26:55.198597425 +0000 UTC m=+1597.972997381"
Oct 30 21:26:55.245369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.245304    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podStartSLOduration=2.245288349 podStartE2EDuration="2.245288349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:55.239581043 +0000 UTC m=+1598.013981017" watchObservedRunningTime="2026-10-30 21:26:55.245288349 +0000 UTC m=+1598.019688285"
Oct 30 21:26:55.478451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.478415    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74e9a27f-b860-4a10-a145-2513437578ad" path="/var/lib/kubelet/pods/74e9a27f-b860-4a10-a145-2513437578ad/volumes"
Oct 30 21:26:55.949012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.948440    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:26:55.949012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:55.948559    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:26:56.034788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.034285    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:26:56.034788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.034433    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:26:56.034788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.034485    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:26:56.040563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.040471    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:56.137882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.137809    4549 generic.go:356] "Generic (PLEG): container finished" podID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerID="e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a" exitCode=0
Oct 30 21:26:56.137882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.137865    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" event={"ID":"6d79a5cf-d13a-4723-9ea2-517205e31827","Type":"ContainerDied","Data":"e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"}
Oct 30 21:26:56.139913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.138399    4549 scope.go:122] "RemoveContainer" containerID="e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"
Oct 30 21:26:56.147530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.140563    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:56.147530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.140636    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:56.147530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.140677    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:56.446011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.434651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:56.625861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.624470    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:56.625861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.624606    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:26:56.642516 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:26:56.642426    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:26:56.953859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.950683    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:26:56.953859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:56.950775    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:26:57.029894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.029548    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:26:57.029894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.029610    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:26:57.613856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.604668    4549 generic.go:356] "Generic (PLEG): container finished" podID="22f11dbc-7817-471f-b839-0b277254e60b" containerID="1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b" exitCode=0
Oct 30 21:26:57.613856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.612446    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" event={"ID":"22f11dbc-7817-471f-b839-0b277254e60b","Type":"ContainerDied","Data":"1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"}
Oct 30 21:26:57.627265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.627227    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" event={"ID":"6d79a5cf-d13a-4723-9ea2-517205e31827","Type":"ContainerStarted","Data":"c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580"}
Oct 30 21:26:57.641859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.640256    4549 scope.go:122] "RemoveContainer" containerID="1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"
Oct 30 21:26:57.645548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.645440    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:57.645850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.645803    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:57.646138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.646114    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:57.689591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.688136    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:26:57.690395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.690362    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:26:57.690690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.690663    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:26:57.948151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.948109    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:26:57.948349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:57.948179    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:26:58.029930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.028288    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:26:58.029930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.028351    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:26:58.183921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.181441    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.664940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.664904    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" event={"ID":"22f11dbc-7817-471f-b839-0b277254e60b","Type":"ContainerStarted","Data":"5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac"}
Oct 30 21:26:58.691857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.677261    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:26:58.691857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.677326    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:26:58.712488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.712423    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:26:58.712711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.712567    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:26:58.712711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.712653    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:26:58.925146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.868007    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.962483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.958928    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:26:58.962483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:58.958985    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:26:59.060560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.053082    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:26:59.060560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.053137    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:26:59.060560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.055969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:59.230266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.230008    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:26:59.765055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.765020    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:26:59.765365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.765339    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:26:59.824204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.824171    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:26:59.824504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.824471    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:26:59.954332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.954293    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:26:59.954628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:26:59.954597    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:27:00.226857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.215557    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:27:00.226857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.215633    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:27:00.226857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.221107    4549 prober.go:122] "Probe failed" probeType="Startup" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.147:8000/health\": context deadline exceeded"
Oct 30 21:27:00.248878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.237538    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:00.284853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.284786    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:00.349897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.344362    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:00.603062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.603001    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded"
Oct 30 21:27:00.626986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.626925    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:00.708997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.708957    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:27:00.709291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.709258    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:27:00.756866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.722204    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:00.756866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.726146    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:27:00.756866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:00.748103    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:01.024922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.020295    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:27:01.024922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.020381    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:27:01.087733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.037439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.165857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.161958    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:01.345022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.344972    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:27:01.479934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.363064    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:27:01.479934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.471341    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:27:01.479934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.427994    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:27:01.479934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.364287    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:27:01.582857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.574085    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.862128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.862092    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:01.862436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.862409    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:01.903097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.903049    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:27:02.001050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:01.953194    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:27:02.331413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.141996    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:02.331413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.163595    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:27:02.331413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.329676    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:27:02.331413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.163512    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.331413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.165385    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:27:02.331413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.328926    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:27:02.377055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.363273    4549 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:02.483870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.475117    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:02.483870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.475572    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:27:02.719915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.716291    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cdl4t" podUID="fd4d565e-8ace-4a8a-a8fb-114a90e3f64c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:02.770558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.770506    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:27:02.770932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.770887    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:27:02.806969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.806917    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:27:02.807292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.807243    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:27:02.817799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.808261    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:02.818012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:02.817848    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:03.037078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.036982    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:27:03.045219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.045173    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:27:03.063587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.063542    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:03.119864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.119785    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:27:03.120198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.120157    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:27:03.368858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.360142    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:27:03.368858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.361628    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 21:27:03.369986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.369936    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:27:03.506322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.506257    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:03.531965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.519982    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded"
Oct 30 21:27:03.650809 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:27:03.650697    4549 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.305s"
Oct 30 21:27:03.651357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.651333    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:27:03.717878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.717844    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:03.721857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.718142    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:27:03.742950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.722135    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:27:03.743979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.718151    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:03.784877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.782181    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.961272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.961177    4549 patch_prober.go:29] interesting pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 21:27:03.969363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.969316    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 21:27:03.994216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.968471    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:27:03.994540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.994521    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:27:03.994691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.994679    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:27:03.994852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.994820    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:27:03.995001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.994989    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:27:03.995134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:03.995123    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:27:04.091928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.078955    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:27:04.156855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.153144    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:27:04.334390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.326697    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:27:04.421022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.420994    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:27:04.455856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.442122    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:27:04.455856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.442182    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:27:04.455856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.442264    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4"
Oct 30 21:27:04.455856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.365918    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:04.509860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.507421    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:27:04.595963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.578056    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:27:04.667477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.667114    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:27:04.718857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.694219    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:27:04.718857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.694283    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:27:04.732907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.730816    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:04.732907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.730944    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:04.955179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.955097    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:27:04.955597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.955574    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:27:04.956500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:04.956479    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:27:05.100045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:05.100008    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:27:05.678275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:05.678237    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:27:05.688859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:05.684104    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:27:05.712870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:05.712805    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:05.713162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:05.713132    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:05.941243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:05.941140    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:06.142025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.141985    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:27:06.142225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.142089    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:27:06.697056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.697019    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:27:06.697545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.697321    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:27:06.698329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.698307    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:27:06.706215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.706187    4549 patch_prober.go:29] interesting pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:27:06.706337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:06.706238    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 21:27:07.354021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:07.353967    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:07.900178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:07.900145    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:27:07.906023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:07.905099    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:27:08.289379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:08.289282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:08.492546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:08.492514    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.080377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:09.080343    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:27:09.092471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:09.092442    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:27:09.160375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:09.160334    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.683137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:09.683107    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:10.146577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.146547    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"]
Oct 30 21:27:10.150191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.150173    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.168768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.168750    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3b28bc-openshift-default-dockercfg-64znn\""
Oct 30 21:27:10.191343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191309    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.191457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191351    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.191510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191496    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.191561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191546    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.191660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191633    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.191795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191772    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.192029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.191988    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.197917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.192059    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr452\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.197917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.192114    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.198554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.198531    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"]
Oct 30 21:27:10.294590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.294556    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.294946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.294920    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.295226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.295209    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.295902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.295880    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.296418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.295661    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.311417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.311361    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr452\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.311417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.311416    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.311656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.311608    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.311656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.311647    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.311799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.311779    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.334853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.312750    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.334853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.313351    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.334853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.313596    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.334853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.313818    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.344854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.336116    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.410910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.387353    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.417854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.412542    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr452\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.442856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.434148    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy\") pod \"e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") " pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.464855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.455734    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:10.471855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.466210    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:10.655027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:10.648356    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:11.091654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.091624    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"]
Oct 30 21:27:11.092400 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:27:11.092378    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b73a33_2d90_4e3d_9680_022ad03c5a2e.slice/crio-25700af00e6e7c66e1912b64f3e54c65c4acca5cc78622960b974a542314c530 WatchSource:0}: Error finding container 25700af00e6e7c66e1912b64f3e54c65c4acca5cc78622960b974a542314c530: Status 404 returned error can't find the container with id 25700af00e6e7c66e1912b64f3e54c65c4acca5cc78622960b974a542314c530
Oct 30 21:27:11.096122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.096084    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:27:11.096235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.096167    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:27:11.096235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.096216    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:27:11.687350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.687313    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" event={"ID":"c4b73a33-2d90-4e3d-9680-022ad03c5a2e","Type":"ContainerStarted","Data":"31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"}
Oct 30 21:27:11.687785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.687384    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" event={"ID":"c4b73a33-2d90-4e3d-9680-022ad03c5a2e","Type":"ContainerStarted","Data":"25700af00e6e7c66e1912b64f3e54c65c4acca5cc78622960b974a542314c530"}
Oct 30 21:27:11.765976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:11.765924    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podStartSLOduration=1.765908509 podStartE2EDuration="1.765908509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:11.752772058 +0000 UTC m=+1614.527171998" watchObservedRunningTime="2026-10-30 21:27:11.765908509 +0000 UTC m=+1614.540308447"
Oct 30 21:27:12.054954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:12.051845    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:12.472765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:12.472729    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:12.479922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:12.479896    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:12.695250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:12.695218    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:12.696199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:12.696176    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:27:14.397544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:14.397485    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:14.615861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:14.610215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:16.177371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.177315    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"]
Oct 30 21:27:16.183823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.183795    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.209425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.209393    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"]
Oct 30 21:27:16.216304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.216276    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-093daa\"/\"openshift-service-ca.crt\""
Oct 30 21:27:16.219057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.219027    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-093daa\"/\"default-dockercfg-mjm95\""
Oct 30 21:27:16.219192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.219062    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-093daa\"/\"kube-root-ca.crt\""
Oct 30 21:27:16.219258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.219225    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-093daa\"/\"test-model-093daa-kserve-self-signed-certs\""
Oct 30 21:27:16.253789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.253714    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.254716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.254686    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.255131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.255102    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.255254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.255207    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.255254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.255241    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.255389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.255274    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvwgr\" (UniqueName: \"kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.295973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.295917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:16.357888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.357856    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.357950    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.357982    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.358016    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvwgr\" (UniqueName: \"kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.358253    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.358297    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.358319    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.358678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.358651    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.359022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.358663    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.360884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.360866    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.361105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.361080    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.398217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.398185    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvwgr\" (UniqueName: \"kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr\") pod \"test-model-093daa-kserve-744f4978f5-4xzjs\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") " pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.510907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.505156    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:16.686886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:16.664691    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:17.193002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:17.192428    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:17.531136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:17.531045    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:17.550736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:17.550698    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"]
Oct 30 21:27:17.563046 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:27:17.563005    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode745e0e6_7871_4b4d_8f3b_241d3a83b4ea.slice/crio-6e6bea77eb7f1937b002eb432591da8bfc0ca29bd1d6e22c9ed60f9335f22d7e WatchSource:0}: Error finding container 6e6bea77eb7f1937b002eb432591da8bfc0ca29bd1d6e22c9ed60f9335f22d7e: Status 404 returned error can't find the container with id 6e6bea77eb7f1937b002eb432591da8bfc0ca29bd1d6e22c9ed60f9335f22d7e
Oct 30 21:27:17.773437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:17.773399    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" event={"ID":"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea","Type":"ContainerStarted","Data":"6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee"}
Oct 30 21:27:17.773621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:17.773444    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" event={"ID":"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea","Type":"ContainerStarted","Data":"6e6bea77eb7f1937b002eb432591da8bfc0ca29bd1d6e22c9ed60f9335f22d7e"}
Oct 30 21:27:17.824343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:17.824285    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" podStartSLOduration=1.8242671019999999 podStartE2EDuration="1.824267102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:17.820050359 +0000 UTC m=+1620.594450299" watchObservedRunningTime="2026-10-30 21:27:17.824267102 +0000 UTC m=+1620.598667063"
Oct 30 21:27:18.505927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:18.505885    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:19.511914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:19.510875    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:21.008778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:21.008744    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.481115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:22.481079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.635851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:22.634396    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:24.249861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:24.248657    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:25.874875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:25.874801    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:26.506078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:26.506047    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:26.518846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:26.518800    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:26.579032 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:27:26.578993    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:27:26.937675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:26.937645    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:27:27.403856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:27.392846    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:28.930334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:28.930299    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:29.118557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:29.118521    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:29.703351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:29.703317    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:29.888458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:29.888423    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:30.820926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:30.815335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:32.588405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:32.588354    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:34.386768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:34.386707    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:36.134650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:36.134595    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:37.845857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:37.842401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:38.047903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:38.047860    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:39.634853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:39.634748    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:41.150795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:41.134024    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:42.902855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:42.895904    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:44.859862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:44.859747    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:46.296516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:46.296479    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:46.706797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:46.706765    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.015428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:47.015347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.787186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:47.787067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:48.245048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.245006    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"]
Oct 30 21:27:48.245238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.245205    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy" containerID="cri-o://91ed29a40c3c258de53ece0a568e4ed5ab4a4e9ab82d448ee49f6e2f9053174a" gracePeriod=30
Oct 30 21:27:48.299227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.299188    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:48.423164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.423129    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"]
Oct 30 21:27:48.423434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.423379    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" containerID="cri-o://fc2a397f78b8edd7a85e323b79f355f48aecfa69ce5c1c5e583e8630cd04101c" gracePeriod=30
Oct 30 21:27:48.442018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.441986    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"]
Oct 30 21:27:48.442230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:48.442205    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy" containerID="cri-o://5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a" gracePeriod=30
Oct 30 21:27:49.488987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:49.488915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:51.568983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:51.566762    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:52.045036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:52.044938    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:52.362669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:52.362616    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:52.768915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:52.768017    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:53.340926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:53.340767    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:54.029165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.029073    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:27:54.030026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.029987    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 21:27:54.202705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.202664    4549 generic.go:356] "Generic (PLEG): container finished" podID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerID="91ed29a40c3c258de53ece0a568e4ed5ab4a4e9ab82d448ee49f6e2f9053174a" exitCode=0
Oct 30 21:27:54.202916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.202746    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" event={"ID":"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99","Type":"ContainerDied","Data":"91ed29a40c3c258de53ece0a568e4ed5ab4a4e9ab82d448ee49f6e2f9053174a"}
Oct 30 21:27:54.202916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.202898    4549 scope.go:122] "RemoveContainer" containerID="a5e666c7a17afe76d3410711d0c7ce0dae7ad2cdf74e7273e3255702cbcd6db9"
Oct 30 21:27:54.779008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.765467    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:54.928198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.886305    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988091    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988173    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988354    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988389    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988450    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988496    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988533    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988654    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:54.990064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:54.988802    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket\") pod \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\" (UID: \"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99\") "
Oct 30 21:27:55.024675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.013936    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.043398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.043303    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.046693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.046659    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:27:55.051999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.051960    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.054474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.054444    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.090087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.085385    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:55.099104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.098790    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.099104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.098842    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.099104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.098858    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.099104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.098872    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.099104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.098884    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.102021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.100078    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "kube-api-access-bsznr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:55.102021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.101939    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.157871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.155108    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:55.157871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.155953    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo" pod "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" (UID: "5c4c4f61-eceb-42c4-bdd0-4e57a010ee99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:27:55.222084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.214459    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsznr\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-kube-api-access-bsznr\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.222084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.214498    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.222084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.214516    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.222084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.214530    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.241088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.239954    4549 generic.go:356] "Generic (PLEG): container finished" podID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerID="fc2a397f78b8edd7a85e323b79f355f48aecfa69ce5c1c5e583e8630cd04101c" exitCode=0
Oct 30 21:27:55.241088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.240089    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" event={"ID":"435a34bd-c5d8-4d7d-bb65-a4ccd612f791","Type":"ContainerDied","Data":"fc2a397f78b8edd7a85e323b79f355f48aecfa69ce5c1c5e583e8630cd04101c"}
Oct 30 21:27:55.241088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.240526    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:27:55.242896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.242177    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"
Oct 30 21:27:55.242896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.242261    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz" event={"ID":"5c4c4f61-eceb-42c4-bdd0-4e57a010ee99","Type":"ContainerDied","Data":"0bae33ae032f16ad0609ab2ce290e616ae86cbd27f76edb7e44f9c974a9789ae"}
Oct 30 21:27:55.242896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.242310    4549 scope.go:122] "RemoveContainer" containerID="91ed29a40c3c258de53ece0a568e4ed5ab4a4e9ab82d448ee49f6e2f9053174a"
Oct 30 21:27:55.249548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.249525    4549 generic.go:356] "Generic (PLEG): container finished" podID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerID="5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a" exitCode=0
Oct 30 21:27:55.250005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.249724    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"
Oct 30 21:27:55.250200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.249756    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" event={"ID":"0da7f759-7cf9-49d9-b745-b2ac768234a9","Type":"ContainerDied","Data":"5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a"}
Oct 30 21:27:55.250276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.250240    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z" event={"ID":"0da7f759-7cf9-49d9-b745-b2ac768234a9","Type":"ContainerDied","Data":"be8481ed01dcb6a2a3eab2a5b6110d792e76e8fb97c248c39cf97077ce361a43"}
Oct 30 21:27:55.264079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.264017    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:27:55.268496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.268474    4549 scope.go:122] "RemoveContainer" containerID="5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a"
Oct 30 21:27:55.327281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.327166    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.327281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.327233    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.327649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.327602    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.327856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.327810    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.328016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.327962    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.328173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.328132    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.328351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.328295    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.328531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.328505    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.328703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.328657    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.328926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.328859    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.329110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.328904    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.329110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.329079    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.329416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.329353    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.329591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.329400    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.329591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.329556    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.329904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.329856    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.330079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.330024    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket\") pod \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\" (UID: \"435a34bd-c5d8-4d7d-bb65-a4ccd612f791\") "
Oct 30 21:27:55.330436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.330387    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert\") pod \"0da7f759-7cf9-49d9-b745-b2ac768234a9\" (UID: \"0da7f759-7cf9-49d9-b745-b2ac768234a9\") "
Oct 30 21:27:55.332098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.332062    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:27:55.332974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.332953    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.341965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.334688    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.342135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.334908    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.342135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.335114    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.342135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.335281    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.342445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.335758    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:27:55.342445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.339197    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.342445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.341201    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.354787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.354716    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:27:55.361240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.359495    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:55.361240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.360216    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "kube-api-access-gqzm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:55.361240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.360299    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:55.361240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.360389    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:27:55.362041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.362003    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "kube-api-access-btndk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:55.362715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.362650    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:55.363860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.363788    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy" pod "435a34bd-c5d8-4d7d-bb65-a4ccd612f791" (UID: "435a34bd-c5d8-4d7d-bb65-a4ccd612f791"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.364110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.364085    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.364955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.364858    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy" pod "0da7f759-7cf9-49d9-b745-b2ac768234a9" (UID: "0da7f759-7cf9-49d9-b745-b2ac768234a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:55.403093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.369328    4549 scope.go:122] "RemoveContainer" containerID="ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436485    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436518    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqzm4\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-kube-api-access-gqzm4\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436534    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436549    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436563    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436576    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436591    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da7f759-7cf9-49d9-b745-b2ac768234a9-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436605    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436618    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436633    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436646    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436660    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436672    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436685    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436698    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436711    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btndk\" (UniqueName: \"kubernetes.io/projected/0da7f759-7cf9-49d9-b745-b2ac768234a9-kube-api-access-btndk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436724    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a34bd-c5d8-4d7d-bb65-a4ccd612f791-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.441910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.436737    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da7f759-7cf9-49d9-b745-b2ac768234a9-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:27:55.462062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.462035    4549 scope.go:122] "RemoveContainer" containerID="5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a"
Oct 30 21:27:55.462673 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:27:55.462645    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a\": container with ID starting with 5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a not found: ID does not exist" containerID="5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a"
Oct 30 21:27:55.462816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.462682    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a"} err="failed to get container status \"5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a\": rpc error: code = NotFound desc = could not find container \"5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a\": container with ID starting with 5539b432d45a847fc268f533936d45415edfe5c170a92c1b274b848f1d69922a not found: ID does not exist"
Oct 30 21:27:55.462816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.462709    4549 scope.go:122] "RemoveContainer" containerID="ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"
Oct 30 21:27:55.463056 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:27:55.463033    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd\": container with ID starting with ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd not found: ID does not exist" containerID="ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"
Oct 30 21:27:55.463140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.463066    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd"} err="failed to get container status \"ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd\": rpc error: code = NotFound desc = could not find container \"ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd\": container with ID starting with ddd5a124a13118bc9d66521c92011f5c0b9d4c29170d3b04876e95220042d3bd not found: ID does not exist"
Oct 30 21:27:55.531843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.531781    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"]
Oct 30 21:27:55.536561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.536476    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz"]
Oct 30 21:27:55.743504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.743376    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"]
Oct 30 21:27:55.781249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.781210    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z"]
Oct 30 21:27:55.802971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:55.797449    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:56.259961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:56.259932    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"
Oct 30 21:27:56.259961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:56.259940    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5" event={"ID":"435a34bd-c5d8-4d7d-bb65-a4ccd612f791","Type":"ContainerDied","Data":"e17ad781ba21cce6bca5f4e5c6bd6486dc7faf5ffbe3fa4ac168c333da96700e"}
Oct 30 21:27:56.260484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:56.260005    4549 scope.go:122] "RemoveContainer" containerID="fc2a397f78b8edd7a85e323b79f355f48aecfa69ce5c1c5e583e8630cd04101c"
Oct 30 21:27:56.362218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:56.362186    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"]
Oct 30 21:27:56.421987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:56.419369    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5"]
Oct 30 21:27:56.736861 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:27:56.732973    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:27:57.353801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:57.353758    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" path="/var/lib/kubelet/pods/0da7f759-7cf9-49d9-b745-b2ac768234a9/volumes"
Oct 30 21:27:57.354389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:57.354371    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" path="/var/lib/kubelet/pods/435a34bd-c5d8-4d7d-bb65-a4ccd612f791/volumes"
Oct 30 21:27:57.354869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:57.354853    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" path="/var/lib/kubelet/pods/5c4c4f61-eceb-42c4-bdd0-4e57a010ee99/volumes"
Oct 30 21:27:57.624906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:57.617692    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:59.447904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:59.445350    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:59.835796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:27:59.835753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:00.242190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:00.242106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:00.559329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:00.559250    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:01.195148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:01.195114    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:01.411170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:01.411077    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:02.605348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:02.605227    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:02.808079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:02.808025    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:28:02.808284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:02.808104    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:28:03.148703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:03.148508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:05.121474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:05.121403    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:06.492695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:06.492657    4549 generic.go:356] "Generic (PLEG): container finished" podID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerID="108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383" exitCode=0
Oct 30 21:28:06.494122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:06.492744    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" event={"ID":"e6ccf846-20db-4a8d-874e-428b2a90d59d","Type":"ContainerDied","Data":"108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"}
Oct 30 21:28:06.494122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:06.493307    4549 scope.go:122] "RemoveContainer" containerID="108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"
Oct 30 21:28:06.495330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:06.495299    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:06.495445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:06.495383    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:06.495445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:06.495433    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:07.117058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.117021    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:07.482451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.482322    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:07.509422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.509380    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" event={"ID":"e6ccf846-20db-4a8d-874e-428b2a90d59d","Type":"ContainerStarted","Data":"420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb"}
Oct 30 21:28:07.511304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.511279    4549 generic.go:356] "Generic (PLEG): container finished" podID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerID="90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e" exitCode=0
Oct 30 21:28:07.511425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.511335    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" event={"ID":"a5540263-fc2a-4fc6-b9c4-7045118690ee","Type":"ContainerDied","Data":"90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"}
Oct 30 21:28:07.511699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.511685    4549 scope.go:122] "RemoveContainer" containerID="90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"
Oct 30 21:28:07.513470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.513435    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:07.513579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.513546    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:07.513645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.513600    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:07.567553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.567183    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:28:07.567714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.567601    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:28:07.567714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:07.567649    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:28:08.528880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:08.526472    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" event={"ID":"a5540263-fc2a-4fc6-b9c4-7045118690ee","Type":"ContainerStarted","Data":"ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455"}
Oct 30 21:28:08.568536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:08.568346    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:28:08.568536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:08.568410    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:28:08.842179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:08.842146    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:28:08.842541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:08.842507    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 21:28:08.842588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:08.842572    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 21:28:09.031857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:09.030141    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:09.567677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:09.567635    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:28:09.568107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:09.567701    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:28:09.841462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:09.841384    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 21:28:09.841679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:09.841657    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 21:28:10.567845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:10.567788    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:28:10.568251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:10.567866    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:28:10.842512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:10.842419    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 21:28:10.842512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:10.842472    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 21:28:11.608372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:11.608336    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:28:11.609665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:11.609634    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:11.610466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:11.610436    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:28:11.841885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:11.841824    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 21:28:11.842071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:11.841912    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 21:28:12.568167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:12.568131    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:28:12.568353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:12.568183    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:28:12.842298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:12.842199    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 21:28:12.842298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:12.842258    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 21:28:13.348911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:13.348880    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:28:13.610406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:13.610306    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:28:13.633070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:13.633036    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:13.639475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:13.639449    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:28:13.842137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:13.842101    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 21:28:13.842308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:13.842166    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 21:28:14.846748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:14.846715    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:28:14.847207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:14.847190    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:28:14.847971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:14.847952    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:28:15.522657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:15.522616    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:18.093531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:18.093495    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:19.894688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:19.894657    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:20.230681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:20.230595    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:21.893853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:21.890907    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:22.588391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:22.588347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:24.063165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:24.063118    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:25.921023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:25.920987    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:26.745436 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:28:26.745401    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:28:28.633505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:28.633461    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:30.331534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:30.331480    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:31.622854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:31.621988    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:31.905910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:31.894592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:32.358345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:32.358293    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:34.626670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:34.626631    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:35.578006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:35.577958    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:28:35.578215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:35.578049    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:28:36.870336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:36.870295    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:38.194109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:38.194069    4549 generic.go:356] "Generic (PLEG): container finished" podID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerID="49ca274d2befbb500fc205c781f0e3ebad381a79abf5c4adbb83ed2f41067b3e" exitCode=0
Oct 30 21:28:38.194500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:38.194309    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" event={"ID":"a88d883a-bb1c-4c36-8a95-6911ac25325d","Type":"ContainerDied","Data":"49ca274d2befbb500fc205c781f0e3ebad381a79abf5c4adbb83ed2f41067b3e"}
Oct 30 21:28:38.194985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:38.194961    4549 scope.go:122] "RemoveContainer" containerID="49ca274d2befbb500fc205c781f0e3ebad381a79abf5c4adbb83ed2f41067b3e"
Oct 30 21:28:38.199879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:38.197812    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:38.199879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:38.197995    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:38.199879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:38.198067    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:39.086476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.082731    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.212970    4549 generic.go:356] "Generic (PLEG): container finished" podID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerID="31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0" exitCode=0
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.213203    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" event={"ID":"c4b73a33-2d90-4e3d-9680-022ad03c5a2e","Type":"ContainerDied","Data":"31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"}
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.213800    4549 scope.go:122] "RemoveContainer" containerID="31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.217776    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" event={"ID":"a88d883a-bb1c-4c36-8a95-6911ac25325d","Type":"ContainerStarted","Data":"af9b19bedcc88ba1ec93b28ef143475c5185aaeba604ed908e2e0f7c14b25b0d"}
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.221651    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.221750    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:39.226855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.221802    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:39.325047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.325003    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:39.733092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:39.733002    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:40.159862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.157942    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:28:40.159862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.158075    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:40.159862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.158254    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:40.207852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.203415    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:40.289917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.289876    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" event={"ID":"c4b73a33-2d90-4e3d-9680-022ad03c5a2e","Type":"ContainerStarted","Data":"61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772"}
Oct 30 21:28:40.767145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.748938    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:28:40.767145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.749146    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:40.767145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.749213    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:40.882956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:40.882919    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:41.163183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:41.163145    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:41.163517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:41.163473    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:41.345807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:41.345778    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:41.492054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:41.491953    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:41.492328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:41.492296    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:41.756913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:41.752620    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:42.157603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:42.157561    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:42.157811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:42.157624    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:42.486359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:42.486256    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:42.487037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:42.486997    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:42.701867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:42.701231    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:28:43.157507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.157474    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:43.157733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.157531    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:43.491421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.491315    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:43.491421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.491387    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:43.580448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.580412    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:43.850315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.850285    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"]
Oct 30 21:28:43.850817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.850777    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" containerID="cri-o://13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60" gracePeriod=30
Oct 30 21:28:43.856767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:43.856737    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:44.151971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:44.151785    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:44.177670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:44.177634    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:44.177987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:44.177957    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:44.507916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:44.490778    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:44.507916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:44.490873    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:44.750635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:44.750600    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.010858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.001679    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.158125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.158095    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:45.158378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.158354    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:45.332650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.332531    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"]
Oct 30 21:28:45.333441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333421    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy"
Oct 30 21:28:45.333441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333439    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333460    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333469    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333477    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333482    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333491    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333496    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333515    4549 cpu_manager.go:414] "containerMap: removing container" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy"
Oct 30 21:28:45.333627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333519    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy"
Oct 30 21:28:45.334012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333662    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="435a34bd-c5d8-4d7d-bb65-a4ccd612f791" containerName="istio-proxy"
Oct 30 21:28:45.334012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333679    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy"
Oct 30 21:28:45.334012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333691    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy"
Oct 30 21:28:45.334012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333699    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c4c4f61-eceb-42c4-bdd0-4e57a010ee99" containerName="istio-proxy"
Oct 30 21:28:45.334012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.333706    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da7f759-7cf9-49d9-b745-b2ac768234a9" containerName="istio-proxy"
Oct 30 21:28:45.337561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.337536    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.420854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.413993    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-67f8e0-openshift-default-dockercfg-xjmbd\""
Oct 30 21:28:45.464855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.460806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.517043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.466526    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.610919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.610805    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751158    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65n2r\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751263    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751382    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751417    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751448    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751527    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751625    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.511117    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.751902    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:45.757867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.479655    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"]
Oct 30 21:28:45.844872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.836887    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.864914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.853553    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.880444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.880409    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.880769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.880735    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65n2r\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.881045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.881017    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.881349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.881329    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.881509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.881493    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.881688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.881632    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.881984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.881953    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.882270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.882238    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.888940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.883684    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.888940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.884369    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.928565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.928520    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.932891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.932204    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:45.934782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:45.934742    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:46.048895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.032153    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:46.060794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.051105    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:46.060794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.060645    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65n2r\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:46.062678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.062640    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo\") pod \"e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") " pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:46.158392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.158298    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 21:28:46.158392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.158358    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 21:28:46.199479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.199222    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:46.260102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.260069    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:46.501072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.474193    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 21:28:46.501072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.474276    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 21:28:46.702507 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:28:46.702476    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c7a62d_0ac8_4dec_97ab_a821fa976299.slice/crio-a025b97ab6f51ea57c88b0d60769e4890f4ed5932c0f9f526cf5d13970dc1b2f WatchSource:0}: Error finding container a025b97ab6f51ea57c88b0d60769e4890f4ed5932c0f9f526cf5d13970dc1b2f: Status 404 returned error can't find the container with id a025b97ab6f51ea57c88b0d60769e4890f4ed5932c0f9f526cf5d13970dc1b2f
Oct 30 21:28:46.703023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.702857    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"]
Oct 30 21:28:46.705603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.705569    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:46.705680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.705638    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:46.705680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:46.705666    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:47.162931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.162898    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:28:47.163328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.163303    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:28:47.164200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.164181    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:28:47.482151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.482071    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:28:47.484356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.484331    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:28:47.516089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.516055    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" event={"ID":"b8c7a62d-0ac8-4dec-97ab-a821fa976299","Type":"ContainerStarted","Data":"77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"}
Oct 30 21:28:47.516288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.516113    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" event={"ID":"b8c7a62d-0ac8-4dec-97ab-a821fa976299","Type":"ContainerStarted","Data":"a025b97ab6f51ea57c88b0d60769e4890f4ed5932c0f9f526cf5d13970dc1b2f"}
Oct 30 21:28:47.740078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:47.739957    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podStartSLOduration=2.73993176 podStartE2EDuration="2.73993176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:47.734244009 +0000 UTC m=+1710.508643962" watchObservedRunningTime="2026-10-30 21:28:47.73993176 +0000 UTC m=+1710.514331723"
Oct 30 21:28:48.223668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.223632    4549 patch_prober.go:29] interesting pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:28:48.223899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.223695    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:28:48.260901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.260871    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:48.266716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.266690    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:48.572957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.563684    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:48.587860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.576237    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:28:48.702857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:48.702379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:49.157643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.157614    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:28:49.228045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228015    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228178    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228343    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228395    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228461    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228509    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228544    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228620    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:28:49.228673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228637    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228719    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs\") pod \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\" (UID: \"7d18cb2f-8a61-494c-9cae-46768da2d8eb\") "
Oct 30 21:28:49.228776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228736    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:49.228776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228759    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:49.228992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228972    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:49.229031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.228988    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:49.229906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.229879    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.230055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.229910    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.230055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.229925    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.230055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.229940    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.230055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.229954    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.231103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.231058    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:28:49.231103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.231102    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:49.231397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.231372    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "kube-api-access-8m4zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:28:49.231495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.231439    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo" pod "7d18cb2f-8a61-494c-9cae-46768da2d8eb" (UID: "7d18cb2f-8a61-494c-9cae-46768da2d8eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:28:49.277074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.277029    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:49.332386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.332355    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.332386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.332392    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.332647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.332403    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d18cb2f-8a61-494c-9cae-46768da2d8eb-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.332647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.332412    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m4zm\" (UniqueName: \"kubernetes.io/projected/7d18cb2f-8a61-494c-9cae-46768da2d8eb-kube-api-access-8m4zm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:28:49.571751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.571712    4549 generic.go:356] "Generic (PLEG): container finished" podID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerID="13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60" exitCode=0
Oct 30 21:28:49.571968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.571770    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"
Oct 30 21:28:49.571968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.571801    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" event={"ID":"7d18cb2f-8a61-494c-9cae-46768da2d8eb","Type":"ContainerDied","Data":"13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60"}
Oct 30 21:28:49.571968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.571905    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s" event={"ID":"7d18cb2f-8a61-494c-9cae-46768da2d8eb","Type":"ContainerDied","Data":"f1216cdce6e179e2f6eb4a9efbb4b835f93b6c74a1563c545c99fa16910178f4"}
Oct 30 21:28:49.571968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.571938    4549 scope.go:122] "RemoveContainer" containerID="13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60"
Oct 30 21:28:49.583601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.583578    4549 scope.go:122] "RemoveContainer" containerID="deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"
Oct 30 21:28:49.594693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.594675    4549 scope.go:122] "RemoveContainer" containerID="13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60"
Oct 30 21:28:49.595017 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:28:49.594998    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60\": container with ID starting with 13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60 not found: ID does not exist" containerID="13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60"
Oct 30 21:28:49.595073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.595026    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60"} err="failed to get container status \"13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60\": rpc error: code = NotFound desc = could not find container \"13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60\": container with ID starting with 13042f8b2ead9e31b1de709b32bc9947ba3df7f1f03968ef59fd2aeb458e6e60 not found: ID does not exist"
Oct 30 21:28:49.595073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.595043    4549 scope.go:122] "RemoveContainer" containerID="deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"
Oct 30 21:28:49.595284 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:28:49.595265    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085\": container with ID starting with deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085 not found: ID does not exist" containerID="deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"
Oct 30 21:28:49.595323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.595292    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085"} err="failed to get container status \"deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085\": rpc error: code = NotFound desc = could not find container \"deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085\": container with ID starting with deadb4011a67e791ef2cae65011fb5574ec7f0f3f276c1e94e14c6b4e5516085 not found: ID does not exist"
Oct 30 21:28:49.640995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.640960    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"]
Oct 30 21:28:49.653112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:49.653083    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s"]
Oct 30 21:28:50.755405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:50.755374    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:51.004857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:51.000498    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:51.349672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:51.349641    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" path="/var/lib/kubelet/pods/7d18cb2f-8a61-494c-9cae-46768da2d8eb/volumes"
Oct 30 21:28:52.649161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:52.649127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:52.890860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:52.887804    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"]
Oct 30 21:28:52.994000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:52.921394    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy"
Oct 30 21:28:52.994000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:52.921425    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy"
Oct 30 21:28:52.994000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:52.922155    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy"
Oct 30 21:28:53.033901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.014311    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.128361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.064557    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"]
Oct 30 21:28:53.220890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.104022    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-67f8e0\""
Oct 30 21:28:53.220890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.104699    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-67f8e0-dockercfg-fq9lg\""
Oct 30 21:28:53.220890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.196774    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.220890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.210057    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tswwl\" (UniqueName: \"kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl\") pod \"payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.232016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.231976    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:53.312603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.312509    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.312603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.312546    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tswwl\" (UniqueName: \"kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl\") pod \"payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.313212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.313191    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.343914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.343885    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tswwl\" (UniqueName: \"kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl\") pod \"payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.445325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.445295    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:53.812204 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:28:53.812167    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4287f93_d448_4133_a8bd_aec42fb593fd.slice/crio-fe2f6da722ab317d5777b3384faf6f16869efdcdd0cda88d754c0f8ae4bb1660 WatchSource:0}: Error finding container fe2f6da722ab317d5777b3384faf6f16869efdcdd0cda88d754c0f8ae4bb1660: Status 404 returned error can't find the container with id fe2f6da722ab317d5777b3384faf6f16869efdcdd0cda88d754c0f8ae4bb1660
Oct 30 21:28:53.812763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.812708    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"]
Oct 30 21:28:53.832779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.832749    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"]
Oct 30 21:28:53.834059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.834038    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy"
Oct 30 21:28:53.834115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.834062    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy"
Oct 30 21:28:53.834261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.834247    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d18cb2f-8a61-494c-9cae-46768da2d8eb" containerName="istio-proxy"
Oct 30 21:28:53.837994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.837979    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:53.862610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.862585    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c9477c-serving-cert\""
Oct 30 21:28:53.876383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:53.876363    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"]
Oct 30 21:28:54.023358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.023322    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.023523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.023501    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.023597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.023562    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.023750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.023728    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9t9v\" (UniqueName: \"kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.135864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.134746    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.135864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.134987    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.135864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.135033    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.135864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.135196    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9t9v\" (UniqueName: \"kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.161852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.152702    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.161852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.161322    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.170854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.167813    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.170854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.168903    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:54.170854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.170279    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9t9v\" (UniqueName: \"kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v\") pod \"maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.188037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.175729    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:54.521440 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:28:54.521406    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ea4b8a_e525_418e_8976_94bb0892c61e.slice/crio-eb0fc9760b20e8e554681372b0762b5f3d29829371ea414c60a759005e3d41e8 WatchSource:0}: Error finding container eb0fc9760b20e8e554681372b0762b5f3d29829371ea414c60a759005e3d41e8: Status 404 returned error can't find the container with id eb0fc9760b20e8e554681372b0762b5f3d29829371ea414c60a759005e3d41e8
Oct 30 21:28:54.523406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.523383    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:28:54.527691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.527668    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"]
Oct 30 21:28:54.614969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.614932    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" event={"ID":"e4287f93-d448-4133-a8bd-aec42fb593fd","Type":"ContainerStarted","Data":"bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4"}
Oct 30 21:28:54.615180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.614998    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" event={"ID":"e4287f93-d448-4133-a8bd-aec42fb593fd","Type":"ContainerStarted","Data":"fe2f6da722ab317d5777b3384faf6f16869efdcdd0cda88d754c0f8ae4bb1660"}
Oct 30 21:28:54.615180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.615031    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:28:54.616451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.616423    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm" event={"ID":"59ea4b8a-e525-418e-8976-94bb0892c61e","Type":"ContainerStarted","Data":"eb0fc9760b20e8e554681372b0762b5f3d29829371ea414c60a759005e3d41e8"}
Oct 30 21:28:54.623580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.623558    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:54.676131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:54.676063    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" podStartSLOduration=2.67604736 podStartE2EDuration="2.67604736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:54.673591108 +0000 UTC m=+1717.447991048" watchObservedRunningTime="2026-10-30 21:28:54.67604736 +0000 UTC m=+1717.450447299"
Oct 30 21:28:55.682721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:55.682682    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm" event={"ID":"59ea4b8a-e525-418e-8976-94bb0892c61e","Type":"ContainerStarted","Data":"79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f"}
Oct 30 21:28:55.683313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:55.682770    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:28:55.731884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:55.731809    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm" podStartSLOduration=2.360413377 podStartE2EDuration="2.731792634s" totalImagesPullingTime="371.379257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:55.729059466 +0000 UTC m=+1718.503459406" watchObservedRunningTime="2026-10-30 21:28:55.731792634 +0000 UTC m=+1718.506192573"
Oct 30 21:28:56.505964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:56.505486    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:56.576418 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:28:56.576381    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:28:58.237360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:28:58.237316    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:00.036886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:00.035117    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:01.696434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:01.696406    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:29:02.165863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:02.165508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:02.436426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:02.436348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:04.200316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:04.200282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:04.477610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:04.477513    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:05.686599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:05.686573    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:29:06.746699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:06.746664    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:07.677798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:07.677762    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:08.455660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:08.455607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:08.757674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:08.757587    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:09.079484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:09.079437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:10.803148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:10.803099    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:11.300905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:11.300878    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:11.543531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:11.543439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:12.454905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:12.454820    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:13.152321 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:13.152282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:13.924142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:13.924106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:15.320851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:15.319372    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:17.364731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:17.364687    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:19.183549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:19.183516    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:21.289795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:21.289759    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:23.759678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:23.759644    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:26.103098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:26.103062    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:26.578554 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:29:26.578513    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:29:27.766865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:27.766789    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:29.473553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:29.473352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:31.421332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:31.413673    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:33.227935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:33.227818    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:34.793645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:34.793606    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:36.330606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:36.330566    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:38.159446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:38.159066    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:39.643094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:39.643053    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:41.081049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:41.081007    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:41.256994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:41.256937    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:43.051627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:43.051595    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:44.620506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:44.620464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:46.139813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:46.139679    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:46.310947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:46.310912    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:47.954855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:47.953647    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:49.771742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:49.771702    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:51.251079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:51.251040    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:51.435857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:51.429235    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:53.366855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:53.362389    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:54.891479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:54.891444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:55.919404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:55.919368    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:56.468536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:56.468452    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:56.584615 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:29:56.584580    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:29:57.586013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:57.585984    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:57.757586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:57.757556    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:58.240855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:58.233753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:59.796569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:29:59.796527    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:00.076852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.075674    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:00.313890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.313737    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"]
Oct 30 21:30:00.344906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.329357    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.346183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.346146    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"]
Oct 30 21:30:00.355343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.355313    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"]
Oct 30 21:30:00.361766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.361041    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tnbrd\""
Oct 30 21:30:00.368915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.361654    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.384129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.384090    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.384339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.384232    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mqfm\" (UniqueName: \"kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm\") pod \"maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.396222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.395988    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"]
Oct 30 21:30:00.450978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.450938    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"]
Oct 30 21:30:00.457620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.457591    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.458926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.458895    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"]
Oct 30 21:30:00.465699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.465656    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.465974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.465958    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"]
Oct 30 21:30:00.470877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.470849    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.486468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.486425    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.486612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.486586    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.486709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.486687    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flhx2\" (UniqueName: \"kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2\") pod \"maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.486772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.486756    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mqfm\" (UniqueName: \"kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm\") pod \"maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.487725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.487699    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.488782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.488756    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"]
Oct 30 21:30:00.500594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.498876    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"]
Oct 30 21:30:00.518047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.515507    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"]
Oct 30 21:30:00.522345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.522312    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"]
Oct 30 21:30:00.524464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.524442    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.530749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.530721    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.532940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.532883    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mqfm\" (UniqueName: \"kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm\") pod \"maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.548611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.548378    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"]
Oct 30 21:30:00.554811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.554773    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"]
Oct 30 21:30:00.588375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588339    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.588563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588405    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.588563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588436    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgnx\" (UniqueName: \"kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx\") pod \"maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.588563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588475    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flhx2\" (UniqueName: \"kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2\") pod \"maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.588563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588520    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.588563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588542    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrxc2\" (UniqueName: \"kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2\") pod \"maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.588943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588587    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.588943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588701    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsjnw\" (UniqueName: \"kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw\") pod \"maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.588943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588715    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm6xp\" (UniqueName: \"kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.588943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588782    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmxll\" (UniqueName: \"kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll\") pod \"maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.588943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588858    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.588943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.588886    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.589935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.589800    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.623139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.623064    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flhx2\" (UniqueName: \"kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2\") pod \"maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.626445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.626417    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"]
Oct 30 21:30:00.635102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.635072    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"]
Oct 30 21:30:00.640204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.640181    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.678056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.678012    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"]
Oct 30 21:30:00.690891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.690849    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.691157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.690920    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgnx\" (UniqueName: \"kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx\") pod \"maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.691157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691029    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.691157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691065    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrxc2\" (UniqueName: \"kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2\") pod \"maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.691157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691116    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.691514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691301    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjnw\" (UniqueName: \"kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw\") pod \"maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.691514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691328    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm6xp\" (UniqueName: \"kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.691514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691366    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgg2j\" (UniqueName: \"kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j\") pod \"maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.691514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691480    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmxll\" (UniqueName: \"kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll\") pod \"maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.691845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691514    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.691845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691602    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.691845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.691633    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.692582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.692555    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.692679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.692641    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.693669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.693643    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.693861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.693816    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.694498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.694475    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.720190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.719680    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm6xp\" (UniqueName: \"kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.720190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.720138    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjnw\" (UniqueName: \"kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw\") pod \"maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.721026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.720538    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrxc2\" (UniqueName: \"kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2\") pod \"maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.721167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.721128    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgnx\" (UniqueName: \"kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx\") pod \"maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.728420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.728392    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmxll\" (UniqueName: \"kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll\") pod \"maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.728576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.728439    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:00.769028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.768986    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:00.790202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.790175    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:00.794034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.793996    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.794255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.794240    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgg2j\" (UniqueName: \"kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j\") pod \"maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.794690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.794670    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.811808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.811776    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:00.825078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.825042    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:00.835678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.829330    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgg2j\" (UniqueName: \"kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j\") pod \"maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:00.881159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.881059    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:00.885646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.885605    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:00.964929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:00.964337    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:01.093751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.093271    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"]
Oct 30 21:30:01.311680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.311639    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"]
Oct 30 21:30:01.314130 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.314092    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5164e9_b267_47b4_ba7f_c9dd1d01e42d.slice/crio-7812b5ba4c53cb9a39c8663b5a7c9d6205fbd9a91e8e1ffda234eed228b1f691 WatchSource:0}: Error finding container 7812b5ba4c53cb9a39c8663b5a7c9d6205fbd9a91e8e1ffda234eed228b1f691: Status 404 returned error can't find the container with id 7812b5ba4c53cb9a39c8663b5a7c9d6205fbd9a91e8e1ffda234eed228b1f691
Oct 30 21:30:01.340262 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.340234    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b29b3d_8305_4169_ab07_fc24c72532c8.slice/crio-495542b918fcdce9b7ba77b1092d5fdda97076fd7adb40ee4e57b1c97c764b38 WatchSource:0}: Error finding container 495542b918fcdce9b7ba77b1092d5fdda97076fd7adb40ee4e57b1c97c764b38: Status 404 returned error can't find the container with id 495542b918fcdce9b7ba77b1092d5fdda97076fd7adb40ee4e57b1c97c764b38
Oct 30 21:30:01.351653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.351621    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"]
Oct 30 21:30:01.358589 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.358558    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b89647_cdd0_41de_a42c_a00209b52633.slice/crio-bd6928dd70ea4b683d8ac3ea58a060932c61cef0053e7a0c7b1541354aca5a81 WatchSource:0}: Error finding container bd6928dd70ea4b683d8ac3ea58a060932c61cef0053e7a0c7b1541354aca5a81: Status 404 returned error can't find the container with id bd6928dd70ea4b683d8ac3ea58a060932c61cef0053e7a0c7b1541354aca5a81
Oct 30 21:30:01.358734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.358713    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"]
Oct 30 21:30:01.465967 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.465920    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a21fff0_9f77_4964_8634_3294cea2d27b.slice/crio-95748a509ae777d785a1f7655efe4bee1fdf89b728fb8cbd4bb53ad42912cc5c WatchSource:0}: Error finding container 95748a509ae777d785a1f7655efe4bee1fdf89b728fb8cbd4bb53ad42912cc5c: Status 404 returned error can't find the container with id 95748a509ae777d785a1f7655efe4bee1fdf89b728fb8cbd4bb53ad42912cc5c
Oct 30 21:30:01.467076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.467049    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"]
Oct 30 21:30:01.480018 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.479987    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0eb40f_2f2f_4db3_ab15_6dc7f1a9f1b9.slice/crio-eac7a7a298b763a67c304f6c18f146ce409d28b0cb1b35c2a61010d59f83b7c4 WatchSource:0}: Error finding container eac7a7a298b763a67c304f6c18f146ce409d28b0cb1b35c2a61010d59f83b7c4: Status 404 returned error can't find the container with id eac7a7a298b763a67c304f6c18f146ce409d28b0cb1b35c2a61010d59f83b7c4
Oct 30 21:30:01.481332 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.481298    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87d57bc_fd1f_4e97_baa4_2fed8c00bce8.slice/crio-69ad1164424c4af2d405d7dc38f388a99d83e9427c47456b6aee7fc0aa3ce297 WatchSource:0}: Error finding container 69ad1164424c4af2d405d7dc38f388a99d83e9427c47456b6aee7fc0aa3ce297: Status 404 returned error can't find the container with id 69ad1164424c4af2d405d7dc38f388a99d83e9427c47456b6aee7fc0aa3ce297
Oct 30 21:30:01.481442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.481345    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"]
Oct 30 21:30:01.483976 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:01.483866    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7d83d9_ec2a_4bfd_baee_34c3b5d4c5b1.slice/crio-2d002d7e9ef2adea3f46a528fa629bc7cb8009e0346a6165fca8db788bea762f WatchSource:0}: Error finding container 2d002d7e9ef2adea3f46a528fa629bc7cb8009e0346a6165fca8db788bea762f: Status 404 returned error can't find the container with id 2d002d7e9ef2adea3f46a528fa629bc7cb8009e0346a6165fca8db788bea762f
Oct 30 21:30:01.488149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.488127    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"]
Oct 30 21:30:01.493010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.492990    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"]
Oct 30 21:30:01.591844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.591807    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerStarted","Data":"ee260a351282af28611061b21187349c310ee5ecdc857727a94b1b508e70ded4"}
Oct 30 21:30:01.593161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.593134    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerStarted","Data":"95748a509ae777d785a1f7655efe4bee1fdf89b728fb8cbd4bb53ad42912cc5c"}
Oct 30 21:30:01.594431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.594408    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerStarted","Data":"bd6928dd70ea4b683d8ac3ea58a060932c61cef0053e7a0c7b1541354aca5a81"}
Oct 30 21:30:01.595747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.595727    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerStarted","Data":"7812b5ba4c53cb9a39c8663b5a7c9d6205fbd9a91e8e1ffda234eed228b1f691"}
Oct 30 21:30:01.597008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.596990    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerStarted","Data":"2d002d7e9ef2adea3f46a528fa629bc7cb8009e0346a6165fca8db788bea762f"}
Oct 30 21:30:01.598239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.598213    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" event={"ID":"58b29b3d-8305-4169-ab07-fc24c72532c8","Type":"ContainerStarted","Data":"495542b918fcdce9b7ba77b1092d5fdda97076fd7adb40ee4e57b1c97c764b38"}
Oct 30 21:30:01.599494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.599474    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerStarted","Data":"69ad1164424c4af2d405d7dc38f388a99d83e9427c47456b6aee7fc0aa3ce297"}
Oct 30 21:30:01.600709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:01.600692    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerStarted","Data":"eac7a7a298b763a67c304f6c18f146ce409d28b0cb1b35c2a61010d59f83b7c4"}
Oct 30 21:30:02.283407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:02.282524    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:02.691029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:02.690986    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:03.221242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.221188    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"]
Oct 30 21:30:03.229671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.229634    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.246544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.243299    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"]
Oct 30 21:30:03.260590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.260559    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c9477c-openshift-default-dockercfg-8hhn7\""
Oct 30 21:30:03.267665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.266948    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.272240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.270525    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"]
Oct 30 21:30:03.293769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.293728    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"]
Oct 30 21:30:03.418857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.418731    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.419085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.418876    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.419085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419026    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.419210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419122    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419158    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419299    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvxpg\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419336    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419417    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m44x4\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.419712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419541    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419586    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419643    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.419931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419812    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.419931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419871    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.420092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.419996    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.420092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.420058    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.420226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.420089    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.420226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.420157    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.420349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.420225    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.522546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522455    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.522546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522513    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522577    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522647    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522676    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522725    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvxpg\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522776    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522900    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m44x4\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.522978    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523053    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523087    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523147    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523168    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523231    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523295    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523312    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523370    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.523569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523372    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.524069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.523977    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.524706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.524678    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.524970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.524944    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.525518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.525075    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.525518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.525483    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.526053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.526006    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.526157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.526061    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.526253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.526238    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.526491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.526444    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.527061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.527019    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.529034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.528987    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.544339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.534349    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.544339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.540106    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.544339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.541946    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.561203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.560574    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m44x4\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.563003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.561946    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.563003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.562281    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvxpg\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.563150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.563096    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-45slr\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:03.569601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.569242    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:03.619099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:03.619060    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:04.010758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.010592    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"]
Oct 30 21:30:04.024817 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:04.019118    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9306875b_38bb_450b_977e_e21d1a78c8c0.slice/crio-64fa4c9fe77cfda4329116ede9965a0e7e7b42a332508921071471af540f4c51 WatchSource:0}: Error finding container 64fa4c9fe77cfda4329116ede9965a0e7e7b42a332508921071471af540f4c51: Status 404 returned error can't find the container with id 64fa4c9fe77cfda4329116ede9965a0e7e7b42a332508921071471af540f4c51
Oct 30 21:30:04.024817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.023479    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:04.024817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.023585    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:04.024817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.023639    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:04.236579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.236542    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"]
Oct 30 21:30:04.240663 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:04.240619    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44015348_bfc8_4a79_be03_7234b658ad3d.slice/crio-24a2be878e38190287cf0fb26454200e1fa233bb314f4fef6a6f5683c021fcb7 WatchSource:0}: Error finding container 24a2be878e38190287cf0fb26454200e1fa233bb314f4fef6a6f5683c021fcb7: Status 404 returned error can't find the container with id 24a2be878e38190287cf0fb26454200e1fa233bb314f4fef6a6f5683c021fcb7
Oct 30 21:30:04.247765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.247722    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:04.247893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.247819    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:04.247893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.247881    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:04.379552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.379318    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:04.679671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.678250    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" event={"ID":"9306875b-38bb-450b-977e-e21d1a78c8c0","Type":"ContainerStarted","Data":"9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c"}
Oct 30 21:30:04.679671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.678320    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" event={"ID":"9306875b-38bb-450b-977e-e21d1a78c8c0","Type":"ContainerStarted","Data":"64fa4c9fe77cfda4329116ede9965a0e7e7b42a332508921071471af540f4c51"}
Oct 30 21:30:04.704270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.704234    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" event={"ID":"44015348-bfc8-4a79-be03-7234b658ad3d","Type":"ContainerStarted","Data":"b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63"}
Oct 30 21:30:04.704877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.704543    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" event={"ID":"44015348-bfc8-4a79-be03-7234b658ad3d","Type":"ContainerStarted","Data":"24a2be878e38190287cf0fb26454200e1fa233bb314f4fef6a6f5683c021fcb7"}
Oct 30 21:30:04.823917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:04.820957    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podStartSLOduration=1.820937944 podStartE2EDuration="1.820937944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:04.752643816 +0000 UTC m=+1787.527043796" watchObservedRunningTime="2026-10-30 21:30:04.820937944 +0000 UTC m=+1787.595337883"
Oct 30 21:30:05.586901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:05.570445    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:05.586901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:05.572738    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:05.586901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:05.572799    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:05.625978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:05.620696    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:05.625978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:05.624709    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:30:05.625978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:05.624772    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 21:30:06.154021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:06.138545    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:06.624941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:06.577358    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:06.624941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:06.577432    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:06.626310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:06.626268    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:30:06.626500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:06.626340    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 21:30:07.590882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:07.575330    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:07.590882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:07.575398    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:07.627693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:07.627200    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:30:07.627693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:07.627264    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 21:30:07.728663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:07.728503    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:08.280226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:08.276482    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:08.587857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:08.577339    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:08.587857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:08.577408    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:08.621045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:08.620998    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:30:08.621330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:08.621284    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 21:30:09.588383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:09.588343    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:09.598101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:09.598059    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:09.623797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:09.623679    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:30:09.623797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:09.623734    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 21:30:09.968041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:09.967943    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:10.592855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:10.573244    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:10.592855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:10.573317    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:10.622620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:10.622212    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:30:10.622620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:10.622266    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 21:30:11.575792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.575313    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:30:11.575792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.575382    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 21:30:11.626158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.626129    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:11.627296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.627236    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:11.628265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.628247    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:30:11.704167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.704115    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podStartSLOduration=8.704094788 podStartE2EDuration="8.704094788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:04.822099497 +0000 UTC m=+1787.596499441" watchObservedRunningTime="2026-10-30 21:30:11.704094788 +0000 UTC m=+1794.478494728"
Oct 30 21:30:11.852694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:11.852553    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:12.579071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:12.579038    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:12.580015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:12.579951    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:12.581264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:12.581239    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:30:13.027893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:13.027776    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerStarted","Data":"b30fdbd0a120bc5991ef6bffcea5c0adcc0f34cf82a7d07a70a64085e2639fa7"}
Oct 30 21:30:13.067212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:13.067146    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" podStartSLOduration=2.264399608 podStartE2EDuration="13.067124762s" totalImagesPullingTime="10.802725154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:13.064796151 +0000 UTC m=+1795.839196090" watchObservedRunningTime="2026-10-30 21:30:13.067124762 +0000 UTC m=+1795.841524702"
Oct 30 21:30:13.445627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:13.445589    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:14.038638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.038529    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerStarted","Data":"f24309064f67081c3193f539ced4b06c38fe788bfd4aa6a8516ecf242c11d10d"}
Oct 30 21:30:14.040661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.040625    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerStarted","Data":"7a8520a7b89e58db7f7981245257af42643f6bb3fdc91dce6776fd63e1362ae1"}
Oct 30 21:30:14.042322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.042291    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerStarted","Data":"e67d34a50e68b21d2faeb5ff1a4ae012cb3e3295b541771ec5422e126856b711"}
Oct 30 21:30:14.044130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.044100    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerStarted","Data":"2737b8b6cb9bbbd07c3ec4661fca27640c392cc6dd98df334e09f2e243a33f6e"}
Oct 30 21:30:14.045878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.045854    4549 generic.go:356] "Generic (PLEG): container finished" podID="35b89647-cdd0-41de-a42c-a00209b52633" containerID="b30fdbd0a120bc5991ef6bffcea5c0adcc0f34cf82a7d07a70a64085e2639fa7" exitCode=60
Oct 30 21:30:14.045996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.045938    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerDied","Data":"b30fdbd0a120bc5991ef6bffcea5c0adcc0f34cf82a7d07a70a64085e2639fa7"}
Oct 30 21:30:14.046178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.046163    4549 scope.go:122] "RemoveContainer" containerID="b30fdbd0a120bc5991ef6bffcea5c0adcc0f34cf82a7d07a70a64085e2639fa7"
Oct 30 21:30:14.047711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.047684    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerStarted","Data":"ec9abde63fcea5122e13e0b9b29e36354191778a3c57e99e2198e6b97bb0a489"}
Oct 30 21:30:14.049749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.049716    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerStarted","Data":"717bf1a300d2d156b83218577999891642a1bf654c8cf71a31230e7e69374324"}
Oct 30 21:30:14.051557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.051539    4549 generic.go:356] "Generic (PLEG): container finished" podID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerID="c64742bab507289c10a82c6f3094d50fde1c669f735141790f6977990d0425e1" exitCode=60
Oct 30 21:30:14.051628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.051608    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" event={"ID":"58b29b3d-8305-4169-ab07-fc24c72532c8","Type":"ContainerDied","Data":"c64742bab507289c10a82c6f3094d50fde1c669f735141790f6977990d0425e1"}
Oct 30 21:30:14.051987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.051968    4549 scope.go:122] "RemoveContainer" containerID="c64742bab507289c10a82c6f3094d50fde1c669f735141790f6977990d0425e1"
Oct 30 21:30:14.078591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.078531    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" podStartSLOduration=2.924502359 podStartE2EDuration="14.078510793s" totalImagesPullingTime="11.154008434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.078411264 +0000 UTC m=+1796.852811205" watchObservedRunningTime="2026-10-30 21:30:14.078510793 +0000 UTC m=+1796.852910734"
Oct 30 21:30:14.122150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.122098    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" podStartSLOduration=2.864533456 podStartE2EDuration="14.122082385s" totalImagesPullingTime="11.257548929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.11655912 +0000 UTC m=+1796.890959061" watchObservedRunningTime="2026-10-30 21:30:14.122082385 +0000 UTC m=+1796.896482357"
Oct 30 21:30:14.247345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.247284    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" podStartSLOduration=2.619496441 podStartE2EDuration="14.247265444s" totalImagesPullingTime="11.627769003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.241517152 +0000 UTC m=+1797.015917092" watchObservedRunningTime="2026-10-30 21:30:14.247265444 +0000 UTC m=+1797.021665384"
Oct 30 21:30:14.282182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.282115    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" podStartSLOduration=2.879134955 podStartE2EDuration="14.282097374s" totalImagesPullingTime="11.402962419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.280878441 +0000 UTC m=+1797.055278380" watchObservedRunningTime="2026-10-30 21:30:14.282097374 +0000 UTC m=+1797.056497314"
Oct 30 21:30:14.322565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.322509    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" podStartSLOduration=3.106196578 podStartE2EDuration="14.322493645s" totalImagesPullingTime="11.216297067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.319135102 +0000 UTC m=+1797.093535041" watchObservedRunningTime="2026-10-30 21:30:14.322493645 +0000 UTC m=+1797.096893588"
Oct 30 21:30:14.361715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:14.361660    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" podStartSLOduration=3.159680887 podStartE2EDuration="14.361645102s" totalImagesPullingTime="11.201964215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.357579618 +0000 UTC m=+1797.131979556" watchObservedRunningTime="2026-10-30 21:30:14.361645102 +0000 UTC m=+1797.136045041"
Oct 30 21:30:15.038190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.038094    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:15.097548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.097503    4549 generic.go:356] "Generic (PLEG): container finished" podID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerID="7a8520a7b89e58db7f7981245257af42643f6bb3fdc91dce6776fd63e1362ae1" exitCode=60
Oct 30 21:30:15.098538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.098506    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerDied","Data":"7a8520a7b89e58db7f7981245257af42643f6bb3fdc91dce6776fd63e1362ae1"}
Oct 30 21:30:15.099294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.099270    4549 scope.go:122] "RemoveContainer" containerID="7a8520a7b89e58db7f7981245257af42643f6bb3fdc91dce6776fd63e1362ae1"
Oct 30 21:30:15.110403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.109059    4549 generic.go:356] "Generic (PLEG): container finished" podID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerID="e67d34a50e68b21d2faeb5ff1a4ae012cb3e3295b541771ec5422e126856b711" exitCode=60
Oct 30 21:30:15.110403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.109171    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerDied","Data":"e67d34a50e68b21d2faeb5ff1a4ae012cb3e3295b541771ec5422e126856b711"}
Oct 30 21:30:15.110403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.109816    4549 scope.go:122] "RemoveContainer" containerID="e67d34a50e68b21d2faeb5ff1a4ae012cb3e3295b541771ec5422e126856b711"
Oct 30 21:30:15.114851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.113311    4549 generic.go:356] "Generic (PLEG): container finished" podID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerID="2737b8b6cb9bbbd07c3ec4661fca27640c392cc6dd98df334e09f2e243a33f6e" exitCode=60
Oct 30 21:30:15.114851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.113407    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerDied","Data":"2737b8b6cb9bbbd07c3ec4661fca27640c392cc6dd98df334e09f2e243a33f6e"}
Oct 30 21:30:15.114851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.113968    4549 scope.go:122] "RemoveContainer" containerID="2737b8b6cb9bbbd07c3ec4661fca27640c392cc6dd98df334e09f2e243a33f6e"
Oct 30 21:30:15.122743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.122707    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerStarted","Data":"1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae"}
Oct 30 21:30:15.165408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.162185    4549 generic.go:356] "Generic (PLEG): container finished" podID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerID="ec9abde63fcea5122e13e0b9b29e36354191778a3c57e99e2198e6b97bb0a489" exitCode=60
Oct 30 21:30:15.165408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.162271    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerDied","Data":"ec9abde63fcea5122e13e0b9b29e36354191778a3c57e99e2198e6b97bb0a489"}
Oct 30 21:30:15.165408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.162695    4549 scope.go:122] "RemoveContainer" containerID="ec9abde63fcea5122e13e0b9b29e36354191778a3c57e99e2198e6b97bb0a489"
Oct 30 21:30:15.168302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.168271    4549 generic.go:356] "Generic (PLEG): container finished" podID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerID="717bf1a300d2d156b83218577999891642a1bf654c8cf71a31230e7e69374324" exitCode=60
Oct 30 21:30:15.168509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.168438    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerDied","Data":"717bf1a300d2d156b83218577999891642a1bf654c8cf71a31230e7e69374324"}
Oct 30 21:30:15.170315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.168955    4549 scope.go:122] "RemoveContainer" containerID="717bf1a300d2d156b83218577999891642a1bf654c8cf71a31230e7e69374324"
Oct 30 21:30:15.172976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.172909    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" event={"ID":"58b29b3d-8305-4169-ab07-fc24c72532c8","Type":"ContainerStarted","Data":"eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9"}
Oct 30 21:30:15.178542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.178509    4549 generic.go:356] "Generic (PLEG): container finished" podID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerID="f24309064f67081c3193f539ced4b06c38fe788bfd4aa6a8516ecf242c11d10d" exitCode=60
Oct 30 21:30:15.179840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.179163    4549 scope.go:122] "RemoveContainer" containerID="f24309064f67081c3193f539ced4b06c38fe788bfd4aa6a8516ecf242c11d10d"
Oct 30 21:30:15.179840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.179185    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerDied","Data":"f24309064f67081c3193f539ced4b06c38fe788bfd4aa6a8516ecf242c11d10d"}
Oct 30 21:30:15.360824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:15.360762    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" podStartSLOduration=4.201467218 podStartE2EDuration="15.360747371s" totalImagesPullingTime="11.159280153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:15.358056654 +0000 UTC m=+1798.132456593" watchObservedRunningTime="2026-10-30 21:30:15.360747371 +0000 UTC m=+1798.135147311"
Oct 30 21:30:16.189911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.189877    4549 generic.go:356] "Generic (PLEG): container finished" podID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerID="eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9" exitCode=60
Oct 30 21:30:16.190377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.189989    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" event={"ID":"58b29b3d-8305-4169-ab07-fc24c72532c8","Type":"ContainerDied","Data":"eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9"}
Oct 30 21:30:16.190377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.190067    4549 scope.go:122] "RemoveContainer" containerID="c64742bab507289c10a82c6f3094d50fde1c669f735141790f6977990d0425e1"
Oct 30 21:30:16.190509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.190379    4549 scope.go:122] "RemoveContainer" containerID="eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9"
Oct 30 21:30:16.190749 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:16.190729    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t_odh-ai-gateway-infra(58b29b3d-8305-4169-ab07-fc24c72532c8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8"
Oct 30 21:30:16.192333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.192310    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerStarted","Data":"abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5"}
Oct 30 21:30:16.194580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.194546    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerStarted","Data":"adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93"}
Oct 30 21:30:16.196993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.196969    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerStarted","Data":"58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113"}
Oct 30 21:30:16.199225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.199202    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerStarted","Data":"34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78"}
Oct 30 21:30:16.201401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.201377    4549 generic.go:356] "Generic (PLEG): container finished" podID="35b89647-cdd0-41de-a42c-a00209b52633" containerID="1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae" exitCode=60
Oct 30 21:30:16.201493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.201454    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerDied","Data":"1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae"}
Oct 30 21:30:16.201708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.201688    4549 scope.go:122] "RemoveContainer" containerID="1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae"
Oct 30 21:30:16.202063 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:16.202027    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn_odh-ai-gateway-infra(35b89647-cdd0-41de-a42c-a00209b52633)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" podUID="35b89647-cdd0-41de-a42c-a00209b52633"
Oct 30 21:30:16.203636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.203613    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerStarted","Data":"5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea"}
Oct 30 21:30:16.205565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.205541    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerStarted","Data":"d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762"}
Oct 30 21:30:16.207244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.207226    4549 scope.go:122] "RemoveContainer" containerID="b30fdbd0a120bc5991ef6bffcea5c0adcc0f34cf82a7d07a70a64085e2639fa7"
Oct 30 21:30:16.601577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:16.601536    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:17.216090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.216037    4549 scope.go:122] "RemoveContainer" containerID="eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9"
Oct 30 21:30:17.216975 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.216586    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t_odh-ai-gateway-infra(58b29b3d-8305-4169-ab07-fc24c72532c8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8"
Oct 30 21:30:17.218705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.218679    4549 generic.go:356] "Generic (PLEG): container finished" podID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerID="abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5" exitCode=60
Oct 30 21:30:17.218904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.218767    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerDied","Data":"abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5"}
Oct 30 21:30:17.218904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.218870    4549 scope.go:122] "RemoveContainer" containerID="f24309064f67081c3193f539ced4b06c38fe788bfd4aa6a8516ecf242c11d10d"
Oct 30 21:30:17.219196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.219175    4549 scope.go:122] "RemoveContainer" containerID="abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5"
Oct 30 21:30:17.219776 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.219751    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd_odh-ai-gateway-infra(b87d57bc-fd1f-4e97-baa4-2fed8c00bce8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8"
Oct 30 21:30:17.221226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.221204    4549 generic.go:356] "Generic (PLEG): container finished" podID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerID="adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93" exitCode=60
Oct 30 21:30:17.221314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.221271    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerDied","Data":"adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93"}
Oct 30 21:30:17.221626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.221608    4549 scope.go:122] "RemoveContainer" containerID="adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93"
Oct 30 21:30:17.222094 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.222068    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb_odh-ai-gateway-infra(5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9"
Oct 30 21:30:17.223367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.223344    4549 generic.go:356] "Generic (PLEG): container finished" podID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerID="58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113" exitCode=60
Oct 30 21:30:17.223445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.223387    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerDied","Data":"58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113"}
Oct 30 21:30:17.223718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.223699    4549 scope.go:122] "RemoveContainer" containerID="58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113"
Oct 30 21:30:17.224064 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.224046    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj_odh-ai-gateway-infra(6bd30360-7dee-4b2d-911b-1e44a0499c9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d"
Oct 30 21:30:17.225855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.225813    4549 generic.go:356] "Generic (PLEG): container finished" podID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerID="34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78" exitCode=60
Oct 30 21:30:17.225953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.225873    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerDied","Data":"34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78"}
Oct 30 21:30:17.226185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.226169    4549 scope.go:122] "RemoveContainer" containerID="34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78"
Oct 30 21:30:17.226502 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.226477    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk_odh-ai-gateway-infra(1a21fff0-9f77-4964-8634-3294cea2d27b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b"
Oct 30 21:30:17.228226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.228206    4549 scope.go:122] "RemoveContainer" containerID="1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae"
Oct 30 21:30:17.228587 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.228558    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn_odh-ai-gateway-infra(35b89647-cdd0-41de-a42c-a00209b52633)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" podUID="35b89647-cdd0-41de-a42c-a00209b52633"
Oct 30 21:30:17.229999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.229974    4549 generic.go:356] "Generic (PLEG): container finished" podID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerID="5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea" exitCode=60
Oct 30 21:30:17.230095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.230060    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerDied","Data":"5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea"}
Oct 30 21:30:17.230305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.230292    4549 scope.go:122] "RemoveContainer" containerID="5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea"
Oct 30 21:30:17.230604 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.230572    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc_odh-ai-gateway-infra(ae5164e9-b267-47b4-ba7f-c9dd1d01e42d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d"
Oct 30 21:30:17.232413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.232393    4549 generic.go:356] "Generic (PLEG): container finished" podID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerID="d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762" exitCode=60
Oct 30 21:30:17.232497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.232433    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerDied","Data":"d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762"}
Oct 30 21:30:17.232716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.232698    4549 scope.go:122] "RemoveContainer" containerID="d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762"
Oct 30 21:30:17.232964 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:17.232946    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj_odh-ai-gateway-infra(0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1"
Oct 30 21:30:17.236457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.236439    4549 scope.go:122] "RemoveContainer" containerID="7a8520a7b89e58db7f7981245257af42643f6bb3fdc91dce6776fd63e1362ae1"
Oct 30 21:30:17.256877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.256840    4549 scope.go:122] "RemoveContainer" containerID="e67d34a50e68b21d2faeb5ff1a4ae012cb3e3295b541771ec5422e126856b711"
Oct 30 21:30:17.272793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.272767    4549 scope.go:122] "RemoveContainer" containerID="2737b8b6cb9bbbd07c3ec4661fca27640c392cc6dd98df334e09f2e243a33f6e"
Oct 30 21:30:17.288407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.288382    4549 scope.go:122] "RemoveContainer" containerID="ec9abde63fcea5122e13e0b9b29e36354191778a3c57e99e2198e6b97bb0a489"
Oct 30 21:30:17.303962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.303937    4549 scope.go:122] "RemoveContainer" containerID="717bf1a300d2d156b83218577999891642a1bf654c8cf71a31230e7e69374324"
Oct 30 21:30:17.402861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.402778    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"]
Oct 30 21:30:17.403083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:17.403058    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy" containerID="cri-o://32089c143348fb668a44ac06925bc27ff2b19596086aa32c98855faac7c053f6" gracePeriod=30
Oct 30 21:30:18.286856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.282475    4549 scope.go:122] "RemoveContainer" containerID="abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5"
Oct 30 21:30:18.286856 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:18.282989    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd_odh-ai-gateway-infra(b87d57bc-fd1f-4e97-baa4-2fed8c00bce8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8"
Oct 30 21:30:18.294986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.291964    4549 scope.go:122] "RemoveContainer" containerID="adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93"
Oct 30 21:30:18.294986 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:18.292445    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb_odh-ai-gateway-infra(5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9"
Oct 30 21:30:18.307467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.299430    4549 scope.go:122] "RemoveContainer" containerID="58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113"
Oct 30 21:30:18.307467 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:18.307070    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj_odh-ai-gateway-infra(6bd30360-7dee-4b2d-911b-1e44a0499c9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d"
Oct 30 21:30:18.372851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.357570    4549 scope.go:122] "RemoveContainer" containerID="34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78"
Oct 30 21:30:18.372851 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:18.358059    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk_odh-ai-gateway-infra(1a21fff0-9f77-4964-8634-3294cea2d27b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b"
Oct 30 21:30:18.372851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.369793    4549 scope.go:122] "RemoveContainer" containerID="5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea"
Oct 30 21:30:18.372851 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:18.370271    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc_odh-ai-gateway-infra(ae5164e9-b267-47b4-ba7f-c9dd1d01e42d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d"
Oct 30 21:30:18.400261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.397576    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:18.401883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.401007    4549 scope.go:122] "RemoveContainer" containerID="d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762"
Oct 30 21:30:18.401883 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:18.401484    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj_odh-ai-gateway-infra(0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1"
Oct 30 21:30:18.445097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.445057    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:30:18.447380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:18.447358    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:30:21.115859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.090314    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.147:8000/ready\": context deadline exceeded"
Oct 30 21:30:21.346039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.345988    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:30:21.460498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.460397    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:30:21.461584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.461553    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:30:21.544903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.528157    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:30:21.780932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.735401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:21.918804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.918760    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"]
Oct 30 21:30:21.919329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:21.919299    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy" containerID="cri-o://7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300" gracePeriod=30
Oct 30 21:30:22.640228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.640031    4549 generic.go:356] "Generic (PLEG): container finished" podID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerID="32089c143348fb668a44ac06925bc27ff2b19596086aa32c98855faac7c053f6" exitCode=0
Oct 30 21:30:22.640228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.640113    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" event={"ID":"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24","Type":"ContainerDied","Data":"32089c143348fb668a44ac06925bc27ff2b19596086aa32c98855faac7c053f6"}
Oct 30 21:30:22.750732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.750706    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:30:22.822841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.822797    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.822927    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.822978    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823015    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823083    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823136    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.823292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823260    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823318    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823286    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.823485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823363    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.823544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823508    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823546    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.823600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823579    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert\") pod \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\" (UID: \"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24\") "
Oct 30 21:30:22.823983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.823963    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:22.824639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.824612    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.824741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.824645    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.824741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.824660    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.824741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.824674    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.824741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.824688    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.825473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.825447    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:22.825597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.825580    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "kube-api-access-wrhtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:22.825659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.825605    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:22.825948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.825930    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy" pod "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" (UID: "d8958b7e-cf5b-44b2-ae11-cb9a4841fb24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.926440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.926405    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrhtt\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-kube-api-access-wrhtt\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.926440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.926441    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.926440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.926455    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:22.926780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:22.926468    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:23.621856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:23.616270    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:23.655318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:23.655226    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"
Oct 30 21:30:23.655318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:23.655244    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs" event={"ID":"d8958b7e-cf5b-44b2-ae11-cb9a4841fb24","Type":"ContainerDied","Data":"97244d63155d833c9034875beaf13a8d76af97616472aca4c46214374d0b5663"}
Oct 30 21:30:23.655318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:23.655303    4549 scope.go:122] "RemoveContainer" containerID="32089c143348fb668a44ac06925bc27ff2b19596086aa32c98855faac7c053f6"
Oct 30 21:30:23.746663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:23.746620    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"]
Oct 30 21:30:23.758027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:23.757990    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs"]
Oct 30 21:30:25.347801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:25.347768    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" path="/var/lib/kubelet/pods/d8958b7e-cf5b-44b2-ae11-cb9a4841fb24/volumes"
Oct 30 21:30:25.402853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:25.402650    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:26.600270 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:26.600231    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:30:27.101403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.100722    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:27.409870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.407637    4549 scope.go:122] "RemoveContainer" containerID="eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9"
Oct 30 21:30:27.503182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.503157    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:30:27.603607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603576    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603806    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603858    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603896    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603909    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603943    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.603986    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604040    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604107    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.604451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604144    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604181    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.604451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604212    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo\") pod \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\" (UID: \"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f\") "
Oct 30 21:30:27.604451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604424    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.604727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.604451    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:27.605915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.605893    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.606064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.606050    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.606247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.606224    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.606363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.606251    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.606363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.606268    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.672425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.672217    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "kube-api-access-stdzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:27.672425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.672326    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:27.682988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.682921    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.683962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.683882    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo" pod "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" (UID: "33d4aa36-2d5e-463b-9b7d-e7825c2abe0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:27.708686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.708653    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stdzs\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-kube-api-access-stdzs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.708820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.708691    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.708820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.708706    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.708820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.708720    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:27.717707 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.717674    4549 generic.go:356] "Generic (PLEG): container finished" podID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerID="7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300" exitCode=0
Oct 30 21:30:27.717821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.717782    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"
Oct 30 21:30:27.717910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.717811    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" event={"ID":"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f","Type":"ContainerDied","Data":"7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300"}
Oct 30 21:30:27.717910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.717891    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r" event={"ID":"33d4aa36-2d5e-463b-9b7d-e7825c2abe0f","Type":"ContainerDied","Data":"3472653bacea894d53a40298d61146631cc8a393f62e5c0ed3ed828c8d9fbaba"}
Oct 30 21:30:27.718011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.717920    4549 scope.go:122] "RemoveContainer" containerID="7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300"
Oct 30 21:30:27.731614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.731593    4549 scope.go:122] "RemoveContainer" containerID="7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300"
Oct 30 21:30:27.731912 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:27.731893    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300\": container with ID starting with 7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300 not found: ID does not exist" containerID="7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300"
Oct 30 21:30:27.731968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.731923    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300"} err="failed to get container status \"7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300\": rpc error: code = NotFound desc = could not find container \"7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300\": container with ID starting with 7c620747fed7ea1ee1d8e9a559af72282584b71f318d3f3eeda8e095b812c300 not found: ID does not exist"
Oct 30 21:30:27.773803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.773763    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"]
Oct 30 21:30:27.785788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:27.782694    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r"]
Oct 30 21:30:28.337314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:28.337287    4549 scope.go:122] "RemoveContainer" containerID="1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae"
Oct 30 21:30:28.644100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:28.643973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:28.773119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:28.773048    4549 generic.go:356] "Generic (PLEG): container finished" podID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerID="0eb654982c234852fa513e23bddd0cebc877437c02219d956426dc9696c5b476" exitCode=60
Oct 30 21:30:28.773693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:28.773651    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" event={"ID":"58b29b3d-8305-4169-ab07-fc24c72532c8","Type":"ContainerDied","Data":"0eb654982c234852fa513e23bddd0cebc877437c02219d956426dc9696c5b476"}
Oct 30 21:30:28.774014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:28.773994    4549 scope.go:122] "RemoveContainer" containerID="eade78ebca47ade9264075981c3841ee4ff00dd1e4be9b3431e18075534258c9"
Oct 30 21:30:28.775179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:28.775157    4549 scope.go:122] "RemoveContainer" containerID="0eb654982c234852fa513e23bddd0cebc877437c02219d956426dc9696c5b476"
Oct 30 21:30:28.775981 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:28.775949    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t_odh-ai-gateway-infra(58b29b3d-8305-4169-ab07-fc24c72532c8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8"
Oct 30 21:30:29.337754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.337719    4549 scope.go:122] "RemoveContainer" containerID="5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea"
Oct 30 21:30:29.338264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.338228    4549 scope.go:122] "RemoveContainer" containerID="34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78"
Oct 30 21:30:29.338723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.338705    4549 scope.go:122] "RemoveContainer" containerID="58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113"
Oct 30 21:30:29.355882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.354151    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" path="/var/lib/kubelet/pods/33d4aa36-2d5e-463b-9b7d-e7825c2abe0f/volumes"
Oct 30 21:30:29.787160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.787132    4549 generic.go:356] "Generic (PLEG): container finished" podID="35b89647-cdd0-41de-a42c-a00209b52633" containerID="d1e763235299ffb73c8c15d9f5f40ab4ba2c1d9f75d5a398c958d1f9e7411068" exitCode=60
Oct 30 21:30:29.787617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.787311    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerDied","Data":"d1e763235299ffb73c8c15d9f5f40ab4ba2c1d9f75d5a398c958d1f9e7411068"}
Oct 30 21:30:29.787617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.787363    4549 scope.go:122] "RemoveContainer" containerID="1f6c9299fb1b2b7287db7415ce8910264c9961aa0dca401d936bb842c3c054ae"
Oct 30 21:30:29.787919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.787901    4549 scope.go:122] "RemoveContainer" containerID="d1e763235299ffb73c8c15d9f5f40ab4ba2c1d9f75d5a398c958d1f9e7411068"
Oct 30 21:30:29.788521 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:29.788425    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn_odh-ai-gateway-infra(35b89647-cdd0-41de-a42c-a00209b52633)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" podUID="35b89647-cdd0-41de-a42c-a00209b52633"
Oct 30 21:30:29.830367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:29.830324    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"]
Oct 30 21:30:30.066693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.066658    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:30.164429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.164330    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca\") pod \"58b29b3d-8305-4169-ab07-fc24c72532c8\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") "
Oct 30 21:30:30.164619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.164559    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j\" (UniqueName: \"kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j\") pod \"58b29b3d-8305-4169-ab07-fc24c72532c8\" (UID: \"58b29b3d-8305-4169-ab07-fc24c72532c8\") "
Oct 30 21:30:30.164820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.164792    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca" pod "58b29b3d-8305-4169-ab07-fc24c72532c8" (UID: "58b29b3d-8305-4169-ab07-fc24c72532c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:30.166108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.166076    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b29b3d-8305-4169-ab07-fc24c72532c8-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:30.167438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.167389    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j" pod "58b29b3d-8305-4169-ab07-fc24c72532c8" (UID: "58b29b3d-8305-4169-ab07-fc24c72532c8"). InnerVolumeSpecName "kube-api-access-mgg2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:30.268373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.268340    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgg2j\" (UniqueName: \"kubernetes.io/projected/58b29b3d-8305-4169-ab07-fc24c72532c8-kube-api-access-mgg2j\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:30.336754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.336724    4549 scope.go:122] "RemoveContainer" containerID="d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762"
Oct 30 21:30:30.337449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.337422    4549 scope.go:122] "RemoveContainer" containerID="abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5"
Oct 30 21:30:30.400855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.392927    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:30.809879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.809842    4549 generic.go:356] "Generic (PLEG): container finished" podID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerID="b703c1cd3d9f2e95a426338a9de8f6b746a281bd544b013f19a79bb877041291" exitCode=60
Oct 30 21:30:30.812147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.810001    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerDied","Data":"b703c1cd3d9f2e95a426338a9de8f6b746a281bd544b013f19a79bb877041291"}
Oct 30 21:30:30.812147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.810690    4549 scope.go:122] "RemoveContainer" containerID="58b498596378af2c5ef6ea85c071f376af21e01d039e43fa171b223b845c6113"
Oct 30 21:30:30.812147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.811452    4549 scope.go:122] "RemoveContainer" containerID="b703c1cd3d9f2e95a426338a9de8f6b746a281bd544b013f19a79bb877041291"
Oct 30 21:30:30.815441 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:30.812771    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj_odh-ai-gateway-infra(6bd30360-7dee-4b2d-911b-1e44a0499c9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d"
Oct 30 21:30:30.815441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.815095    4549 generic.go:356] "Generic (PLEG): container finished" podID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerID="bd4bc23c5ddcbec6d79c271f03db1677ee16dca4a232a799234095f483d9530e" exitCode=60
Oct 30 21:30:30.815441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.815196    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerDied","Data":"bd4bc23c5ddcbec6d79c271f03db1677ee16dca4a232a799234095f483d9530e"}
Oct 30 21:30:30.815755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.815732    4549 scope.go:122] "RemoveContainer" containerID="bd4bc23c5ddcbec6d79c271f03db1677ee16dca4a232a799234095f483d9530e"
Oct 30 21:30:30.816369 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:30.816340    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk_odh-ai-gateway-infra(1a21fff0-9f77-4964-8634-3294cea2d27b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b"
Oct 30 21:30:30.822973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.822935    4549 generic.go:356] "Generic (PLEG): container finished" podID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerID="612d7b20e6e827aed049f407efe44e347fc1d8684318386826f6b52c51516f54" exitCode=60
Oct 30 21:30:30.823105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.823023    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerDied","Data":"612d7b20e6e827aed049f407efe44e347fc1d8684318386826f6b52c51516f54"}
Oct 30 21:30:30.823512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.823494    4549 scope.go:122] "RemoveContainer" containerID="612d7b20e6e827aed049f407efe44e347fc1d8684318386826f6b52c51516f54"
Oct 30 21:30:30.823889 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:30.823861    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc_odh-ai-gateway-infra(ae5164e9-b267-47b4-ba7f-c9dd1d01e42d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d"
Oct 30 21:30:30.824673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.824656    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"
Oct 30 21:30:30.824762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.824716    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t" event={"ID":"58b29b3d-8305-4169-ab07-fc24c72532c8","Type":"ContainerDied","Data":"495542b918fcdce9b7ba77b1092d5fdda97076fd7adb40ee4e57b1c97c764b38"}
Oct 30 21:30:30.840812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.840785    4549 scope.go:122] "RemoveContainer" containerID="34bd29208f6feb3bedc7ad422271b132f982a1ba48d334a091cdc1028e751f78"
Oct 30 21:30:30.844804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.844782    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"]
Oct 30 21:30:30.861278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.861249    4549 scope.go:122] "RemoveContainer" containerID="5fa80236a67e273da2aaa5d9c164adb13ae5fd53844575cf0389792afa4b4dea"
Oct 30 21:30:30.878163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.878130    4549 scope.go:122] "RemoveContainer" containerID="0eb654982c234852fa513e23bddd0cebc877437c02219d956426dc9696c5b476"
Oct 30 21:30:30.987899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.987861    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"]
Oct 30 21:30:30.995919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:30.995889    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b2628a-29889930-slh9t"]
Oct 30 21:30:31.014201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.014173    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:31.146320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.146280    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2\" (UniqueName: \"kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2\") pod \"35b89647-cdd0-41de-a42c-a00209b52633\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") "
Oct 30 21:30:31.146535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.146343    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca\") pod \"35b89647-cdd0-41de-a42c-a00209b52633\" (UID: \"35b89647-cdd0-41de-a42c-a00209b52633\") "
Oct 30 21:30:31.149327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.149289    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca" pod "35b89647-cdd0-41de-a42c-a00209b52633" (UID: "35b89647-cdd0-41de-a42c-a00209b52633"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:31.151299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.151270    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2" pod "35b89647-cdd0-41de-a42c-a00209b52633" (UID: "35b89647-cdd0-41de-a42c-a00209b52633"). InnerVolumeSpecName "kube-api-access-qrxc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:31.250419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.250320    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrxc2\" (UniqueName: \"kubernetes.io/projected/35b89647-cdd0-41de-a42c-a00209b52633-kube-api-access-qrxc2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:31.250419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.250352    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b89647-cdd0-41de-a42c-a00209b52633-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:31.336679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.336638    4549 scope.go:122] "RemoveContainer" containerID="adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93"
Oct 30 21:30:31.353875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.353842    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" path="/var/lib/kubelet/pods/58b29b3d-8305-4169-ab07-fc24c72532c8/volumes"
Oct 30 21:30:31.469108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.469068    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"]
Oct 30 21:30:31.478872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.477930    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"]
Oct 30 21:30:31.482732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.482700    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"]
Oct 30 21:30:31.838038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.837945    4549 generic.go:356] "Generic (PLEG): container finished" podID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerID="8cd85f8e0f423f5dd3b4cffcb07317754488177912dc6306f011299ecfe4279a" exitCode=60
Oct 30 21:30:31.838518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.838026    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerDied","Data":"8cd85f8e0f423f5dd3b4cffcb07317754488177912dc6306f011299ecfe4279a"}
Oct 30 21:30:31.838518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.838094    4549 scope.go:122] "RemoveContainer" containerID="abb10bc34ca3291c0c3d926e66617db5627d50f2505015b7278b12703d0630d5"
Oct 30 21:30:31.838670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.838650    4549 scope.go:122] "RemoveContainer" containerID="8cd85f8e0f423f5dd3b4cffcb07317754488177912dc6306f011299ecfe4279a"
Oct 30 21:30:31.839258 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:31.839185    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd_odh-ai-gateway-infra(b87d57bc-fd1f-4e97-baa4-2fed8c00bce8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8"
Oct 30 21:30:31.840690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.840661    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerStarted","Data":"1a546207498b1a610cab9f854f608e5594bb4bb2f819d77361c11bdcdf22269d"}
Oct 30 21:30:31.847317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.847256    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn" event={"ID":"35b89647-cdd0-41de-a42c-a00209b52633","Type":"ContainerDied","Data":"bd6928dd70ea4b683d8ac3ea58a060932c61cef0053e7a0c7b1541354aca5a81"}
Oct 30 21:30:31.848061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.848032    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"
Oct 30 21:30:31.853545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.853466    4549 generic.go:356] "Generic (PLEG): container finished" podID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerID="a8e9b31c1c6575d426480dbb245f3b950175d8791c977ea15ce6b27470305856" exitCode=60
Oct 30 21:30:31.853707 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.853683    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerDied","Data":"a8e9b31c1c6575d426480dbb245f3b950175d8791c977ea15ce6b27470305856"}
Oct 30 21:30:31.854308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.854284    4549 scope.go:122] "RemoveContainer" containerID="a8e9b31c1c6575d426480dbb245f3b950175d8791c977ea15ce6b27470305856"
Oct 30 21:30:31.854863 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:31.854786    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj_odh-ai-gateway-infra(0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1"
Oct 30 21:30:31.857069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.857045    4549 scope.go:122] "RemoveContainer" containerID="d1e763235299ffb73c8c15d9f5f40ab4ba2c1d9f75d5a398c958d1f9e7411068"
Oct 30 21:30:31.886349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.883030    4549 scope.go:122] "RemoveContainer" containerID="d9fce1515bea605745ee3d035d234c9b0e2465799cd1af2e196d8b079bb80762"
Oct 30 21:30:31.987903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:31.983164    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:32.076242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.076203    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"]
Oct 30 21:30:32.088605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.088497    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c31999-29889930-bbppn"]
Oct 30 21:30:32.205547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.205475    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:32.269535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.269504    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:32.274713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.274671    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:32.395526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.395489    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2\" (UniqueName: \"kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2\") pod \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") "
Oct 30 21:30:32.395684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.395566    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca\") pod \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") "
Oct 30 21:30:32.395684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.395658    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx\" (UniqueName: \"kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx\") pod \"1a21fff0-9f77-4964-8634-3294cea2d27b\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") "
Oct 30 21:30:32.396397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.396361    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm\" (UniqueName: \"kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm\") pod \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\" (UID: \"6bd30360-7dee-4b2d-911b-1e44a0499c9d\") "
Oct 30 21:30:32.396674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.396645    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca\") pod \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\" (UID: \"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d\") "
Oct 30 21:30:32.396819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.396700    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca" pod "6bd30360-7dee-4b2d-911b-1e44a0499c9d" (UID: "6bd30360-7dee-4b2d-911b-1e44a0499c9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:32.397059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.397034    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca" pod "ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" (UID: "ae5164e9-b267-47b4-ba7f-c9dd1d01e42d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:32.397179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.397164    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca\") pod \"1a21fff0-9f77-4964-8634-3294cea2d27b\" (UID: \"1a21fff0-9f77-4964-8634-3294cea2d27b\") "
Oct 30 21:30:32.397641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.397614    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca" pod "1a21fff0-9f77-4964-8634-3294cea2d27b" (UID: "1a21fff0-9f77-4964-8634-3294cea2d27b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:32.399023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.398999    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:32.399247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.399231    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a21fff0-9f77-4964-8634-3294cea2d27b-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:32.399405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.399390    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd30360-7dee-4b2d-911b-1e44a0499c9d-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:32.400391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.400364    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm" pod "6bd30360-7dee-4b2d-911b-1e44a0499c9d" (UID: "6bd30360-7dee-4b2d-911b-1e44a0499c9d"). InnerVolumeSpecName "kube-api-access-5mqfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:32.402078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.402035    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2" pod "ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" (UID: "ae5164e9-b267-47b4-ba7f-c9dd1d01e42d"). InnerVolumeSpecName "kube-api-access-flhx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:32.409713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.409677    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx" pod "1a21fff0-9f77-4964-8634-3294cea2d27b" (UID: "1a21fff0-9f77-4964-8634-3294cea2d27b"). InnerVolumeSpecName "kube-api-access-kkgnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:32.477956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.477917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:32.501465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.501428    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mqfm\" (UniqueName: \"kubernetes.io/projected/6bd30360-7dee-4b2d-911b-1e44a0499c9d-kube-api-access-5mqfm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:32.501465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.501465    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flhx2\" (UniqueName: \"kubernetes.io/projected/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d-kube-api-access-flhx2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:32.501654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.501475    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkgnx\" (UniqueName: \"kubernetes.io/projected/1a21fff0-9f77-4964-8634-3294cea2d27b-kube-api-access-kkgnx\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:32.885126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.885096    4549 generic.go:356] "Generic (PLEG): container finished" podID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerID="1a546207498b1a610cab9f854f608e5594bb4bb2f819d77361c11bdcdf22269d" exitCode=60
Oct 30 21:30:32.886513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.885603    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerDied","Data":"1a546207498b1a610cab9f854f608e5594bb4bb2f819d77361c11bdcdf22269d"}
Oct 30 21:30:32.886513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.885695    4549 scope.go:122] "RemoveContainer" containerID="adfa02b88a168b5713d78206f0cf25fc8149cd1551df84c72b06fa87b1a7aa93"
Oct 30 21:30:32.886513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.886290    4549 scope.go:122] "RemoveContainer" containerID="1a546207498b1a610cab9f854f608e5594bb4bb2f819d77361c11bdcdf22269d"
Oct 30 21:30:32.887590 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:32.887564    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb_odh-ai-gateway-infra(5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9"
Oct 30 21:30:32.893476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.892466    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj" event={"ID":"6bd30360-7dee-4b2d-911b-1e44a0499c9d","Type":"ContainerDied","Data":"ee260a351282af28611061b21187349c310ee5ecdc857727a94b1b508e70ded4"}
Oct 30 21:30:32.893476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.892602    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"
Oct 30 21:30:32.896230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.896191    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk" event={"ID":"1a21fff0-9f77-4964-8634-3294cea2d27b","Type":"ContainerDied","Data":"95748a509ae777d785a1f7655efe4bee1fdf89b728fb8cbd4bb53ad42912cc5c"}
Oct 30 21:30:32.896411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.896356    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"
Oct 30 21:30:32.900607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.900577    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc" event={"ID":"ae5164e9-b267-47b4-ba7f-c9dd1d01e42d","Type":"ContainerDied","Data":"7812b5ba4c53cb9a39c8663b5a7c9d6205fbd9a91e8e1ffda234eed228b1f691"}
Oct 30 21:30:32.900891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.900870    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"
Oct 30 21:30:32.911012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.910984    4549 scope.go:122] "RemoveContainer" containerID="b703c1cd3d9f2e95a426338a9de8f6b746a281bd544b013f19a79bb877041291"
Oct 30 21:30:32.924241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.924208    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"]
Oct 30 21:30:32.960915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.960876    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"]
Oct 30 21:30:32.987251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:32.982304    4549 scope.go:122] "RemoveContainer" containerID="bd4bc23c5ddcbec6d79c271f03db1677ee16dca4a232a799234095f483d9530e"
Oct 30 21:30:33.002243 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.001937    4549 scope.go:122] "RemoveContainer" containerID="612d7b20e6e827aed049f407efe44e347fc1d8684318386826f6b52c51516f54"
Oct 30 21:30:33.124563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.124530    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:33.143553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.143459    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"]
Oct 30 21:30:33.149974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.149944    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-abf04b-29889930-gfgbj"]
Oct 30 21:30:33.176303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.176265    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"]
Oct 30 21:30:33.188459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.188435    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"]
Oct 30 21:30:33.191223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.191137    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-2cda44-29889930-xf4wc"]
Oct 30 21:30:33.196599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.196575    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:33.217040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.217003    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca\") pod \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") "
Oct 30 21:30:33.217227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.217051    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll\" (UniqueName: \"kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll\") pod \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\" (UID: \"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8\") "
Oct 30 21:30:33.217302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.217287    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp\" (UniqueName: \"kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp\") pod \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") "
Oct 30 21:30:33.217407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.217380    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca" pod "b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" (UID: "b87d57bc-fd1f-4e97-baa4-2fed8c00bce8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:33.217485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.217406    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca\") pod \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\" (UID: \"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1\") "
Oct 30 21:30:33.217822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.217794    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca" pod "0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" (UID: "0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:33.218185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.218141    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:33.218185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.218161    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:33.219758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.219732    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll" pod "b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" (UID: "b87d57bc-fd1f-4e97-baa4-2fed8c00bce8"). InnerVolumeSpecName "kube-api-access-xmxll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:33.219886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.219766    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp" pod "0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" (UID: "0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1"). InnerVolumeSpecName "kube-api-access-zm6xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:33.234659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.234626    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"]
Oct 30 21:30:33.241226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.241199    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-43c070-29889930-7xctk"]
Oct 30 21:30:33.320586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.320529    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm6xp\" (UniqueName: \"kubernetes.io/projected/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1-kube-api-access-zm6xp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:33.320888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.320813    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmxll\" (UniqueName: \"kubernetes.io/projected/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8-kube-api-access-xmxll\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:33.347985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.347948    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" path="/var/lib/kubelet/pods/1a21fff0-9f77-4964-8634-3294cea2d27b/volumes"
Oct 30 21:30:33.348913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.348887    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35b89647-cdd0-41de-a42c-a00209b52633" path="/var/lib/kubelet/pods/35b89647-cdd0-41de-a42c-a00209b52633/volumes"
Oct 30 21:30:33.349490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.349470    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" path="/var/lib/kubelet/pods/6bd30360-7dee-4b2d-911b-1e44a0499c9d/volumes"
Oct 30 21:30:33.350742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.350709    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" path="/var/lib/kubelet/pods/ae5164e9-b267-47b4-ba7f-c9dd1d01e42d/volumes"
Oct 30 21:30:33.588380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.587968    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:33.588380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.588050    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:33.833086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.833053    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.964672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.964543    4549 generic.go:356] "Generic (PLEG): container finished" podID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerID="77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899" exitCode=0
Oct 30 21:30:33.965170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.964690    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" event={"ID":"b8c7a62d-0ac8-4dec-97ab-a821fa976299","Type":"ContainerDied","Data":"77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"}
Oct 30 21:30:33.966487 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.966447    4549 scope.go:122] "RemoveContainer" containerID="77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"
Oct 30 21:30:33.969222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.969204    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"
Oct 30 21:30:33.969495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.969455    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:33.969578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.969303    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd" event={"ID":"b87d57bc-fd1f-4e97-baa4-2fed8c00bce8","Type":"ContainerDied","Data":"69ad1164424c4af2d405d7dc38f388a99d83e9427c47456b6aee7fc0aa3ce297"}
Oct 30 21:30:33.969578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.969558    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:33.971369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.969595    4549 scope.go:122] "RemoveContainer" containerID="8cd85f8e0f423f5dd3b4cffcb07317754488177912dc6306f011299ecfe4279a"
Oct 30 21:30:33.971369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.969611    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:33.981240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.981216    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"
Oct 30 21:30:33.981411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.981236    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj" event={"ID":"0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1","Type":"ContainerDied","Data":"2d002d7e9ef2adea3f46a528fa629bc7cb8009e0346a6165fca8db788bea762f"}
Oct 30 21:30:33.995010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:33.994105    4549 scope.go:122] "RemoveContainer" containerID="a8e9b31c1c6575d426480dbb245f3b950175d8791c977ea15ce6b27470305856"
Oct 30 21:30:34.108546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.108481    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"]
Oct 30 21:30:34.123785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.123742    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-79477e-29889930-7v5gd"]
Oct 30 21:30:34.163848 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.163786    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"]
Oct 30 21:30:34.173630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.173595    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-95692f-29889930-mq8rj"]
Oct 30 21:30:34.242350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.242268    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:34.352852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.352719    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca\") pod \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") "
Oct 30 21:30:34.353057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.352904    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw\" (UniqueName: \"kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw\") pod \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\" (UID: \"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9\") "
Oct 30 21:30:34.354884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.354338    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca" pod "5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" (UID: "5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:34.366012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.365978    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw" pod "5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" (UID: "5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9"). InnerVolumeSpecName "kube-api-access-zsjnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:34.455499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.455464    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:34.455499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.455494    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsjnw\" (UniqueName: \"kubernetes.io/projected/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9-kube-api-access-zsjnw\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:30:34.990903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.990865    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" event={"ID":"b8c7a62d-0ac8-4dec-97ab-a821fa976299","Type":"ContainerStarted","Data":"2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365"}
Oct 30 21:30:34.993484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.993460    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"
Oct 30 21:30:34.993643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.993484    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb" event={"ID":"5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9","Type":"ContainerDied","Data":"eac7a7a298b763a67c304f6c18f146ce409d28b0cb1b35c2a61010d59f83b7c4"}
Oct 30 21:30:34.993643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:34.993542    4549 scope.go:122] "RemoveContainer" containerID="1a546207498b1a610cab9f854f608e5594bb4bb2f819d77361c11bdcdf22269d"
Oct 30 21:30:35.086680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.086648    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"]
Oct 30 21:30:35.095965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.095934    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-79c24a-29889930-kggkb"]
Oct 30 21:30:35.261450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.261361    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:30:35.261848 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.261806    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:35.262077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.262047    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:35.454854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.453900    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" path="/var/lib/kubelet/pods/0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1/volumes"
Oct 30 21:30:35.454854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.454664    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" path="/var/lib/kubelet/pods/5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9/volumes"
Oct 30 21:30:35.466855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.455314    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" path="/var/lib/kubelet/pods/b87d57bc-fd1f-4e97-baa4-2fed8c00bce8/volumes"
Oct 30 21:30:35.757678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:35.757631    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:36.261057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:36.261022    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:36.261465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:36.261078    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:37.261270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:37.261237    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:37.261694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:37.261293    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:37.508141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:37.508103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:38.106856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:38.102185    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:38.260535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:38.260502    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:38.260723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:38.260589    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:39.240738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:39.240703    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:39.281855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:39.278336    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:39.281855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:39.278490    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:40.260920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:40.260880    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:40.261329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:40.260945    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:40.595948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:40.595780    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:41.297067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:41.297029    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Oct 30 21:30:41.297901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:41.297860    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Oct 30 21:30:41.301554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:41.301523    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:42.266542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:42.266509    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:30:42.267103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:42.267079    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:30:42.268024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:42.268003    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:30:42.993903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:42.993781    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:43.492716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:43.492682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.021135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.021099    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.438291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.438262    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.476327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.476292    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"]
Oct 30 21:30:44.477500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477482    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:44.477563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477504    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:44.477563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477533    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:44.477563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477541    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:44.477563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477555    4549 cpu_manager.go:414] "containerMap: removing container" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477565    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477594    4549 cpu_manager.go:414] "containerMap: removing container" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477601    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477610    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477616    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477633    4549 cpu_manager.go:414] "containerMap: removing container" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477641    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477655    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477662    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477675    4549 cpu_manager.go:414] "containerMap: removing container" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477683    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477695    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477702    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:44.477710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477713    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477719    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477729    4549 cpu_manager.go:414] "containerMap: removing container" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477736    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477747    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477754    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477773    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477779    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477790    4549 cpu_manager.go:414] "containerMap: removing container" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477796    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477806    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477824    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477856    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477863    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477878    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477885    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477898    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477905    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477920    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477926    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477936    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477943    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477961    4549 cpu_manager.go:414] "containerMap: removing container" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.477968    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478122    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478134    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478144    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478155    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478168    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478177    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:44.478180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478186    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8958b7e-cf5b-44b2-ae11-cb9a4841fb24" containerName="istio-proxy"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478196    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478204    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478213    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478220    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478229    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478238    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478246    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478255    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478263    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478272    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a21fff0-9f77-4964-8634-3294cea2d27b" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478282    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478291    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b87d57bc-fd1f-4e97-baa4-2fed8c00bce8" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478300    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478309    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:44.479152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.478318    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="33d4aa36-2d5e-463b-9b7d-e7825c2abe0f" containerName="istio-proxy"
Oct 30 21:30:44.486245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.486219    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.506387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.506357    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"]
Oct 30 21:30:44.511965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.511941    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f23eae-openshift-default-dockercfg-dk46s\""
Oct 30 21:30:44.669649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.669608    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.670785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.670636    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.670785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.670687    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.671023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.670889    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.671023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.670992    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.671147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.671054    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.671214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.671156    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.693903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.671293    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.694327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.694297    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.694589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.694563    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb498\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800443    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800538    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800591    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800670    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800777    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800816    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800914    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb498\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.800996    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.801031    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.803855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.801547    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.811681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.811633    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.813354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.813330    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.814000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.813977    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.814803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.814785    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.826157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.824820    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.845601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.845566    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.861008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.860961    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.875632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.875598    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:44.881629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:44.881561    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb498\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498\") pod \"e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") " pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:45.116544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.116510    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:45.451377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.451347    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"]
Oct 30 21:30:45.452072 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:45.452048    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a250b6_26b3_4c9c_9c5a_71c3da498cbe.slice/crio-4f55faea2b39261005e5acc5519652189d9236fa3d8d4d7a2954cc9d8b4a8ae1 WatchSource:0}: Error finding container 4f55faea2b39261005e5acc5519652189d9236fa3d8d4d7a2954cc9d8b4a8ae1: Status 404 returned error can't find the container with id 4f55faea2b39261005e5acc5519652189d9236fa3d8d4d7a2954cc9d8b4a8ae1
Oct 30 21:30:45.455110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.455074    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:45.455202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.455142    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:45.455202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.455174    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:45.487188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.487147    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"]
Oct 30 21:30:45.488513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488483    4549 cpu_manager.go:414] "containerMap: removing container" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:45.488513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488511    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58b29b3d-8305-4169-ab07-fc24c72532c8" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488546    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488554    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae5164e9-b267-47b4-ba7f-c9dd1d01e42d" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488569    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488576    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488604    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488611    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488667    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:45.488717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488673    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:45.489165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488892    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f0eb40f-2f2f-4db3-ab15-6dc7f1a9f1b9" containerName="cleanup"
Oct 30 21:30:45.489165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488919    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="35b89647-cdd0-41de-a42c-a00209b52633" containerName="cleanup"
Oct 30 21:30:45.489165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488930    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bd30360-7dee-4b2d-911b-1e44a0499c9d" containerName="cleanup"
Oct 30 21:30:45.489165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.488945    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f7d83d9-ec2a-4bfd-baee-34c3b5d4c5b1" containerName="cleanup"
Oct 30 21:30:45.495448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.495426    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.510614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.510591    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-200d11\"/\"openshift-service-ca.crt\""
Oct 30 21:30:45.510909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.510889    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-200d11\"/\"kube-root-ca.crt\""
Oct 30 21:30:45.512469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.512447    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"]
Oct 30 21:30:45.519980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.519942    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.520159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.520041    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.520159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.520117    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.520269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.520238    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhl9z\" (UniqueName: \"kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.520323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.520286    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.520533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.520517    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.548319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.548287    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-200d11\"/\"test-model-200d11-kserve-self-signed-certs\""
Oct 30 21:30:45.548434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.548339    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-200d11\"/\"default-dockercfg-84gdf\""
Oct 30 21:30:45.559520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.559484    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:45.621921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.621888    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.621960    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622013    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622035    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622107    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhl9z\" (UniqueName: \"kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622150    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622368    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622423    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.622486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.622465    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.624683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.624655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.625144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.625008    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.642639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.642606    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhl9z\" (UniqueName: \"kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z\") pod \"test-model-200d11-kserve-748cb55d75-rmmfh\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") " pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:45.825860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:45.824167    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:46.149496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:46.148978    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" event={"ID":"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe","Type":"ContainerStarted","Data":"6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"}
Oct 30 21:30:46.149496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:46.149037    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" event={"ID":"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe","Type":"ContainerStarted","Data":"4f55faea2b39261005e5acc5519652189d9236fa3d8d4d7a2954cc9d8b4a8ae1"}
Oct 30 21:30:46.198705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:46.198648    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podStartSLOduration=2.198631958 podStartE2EDuration="2.198631958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:46.191770817 +0000 UTC m=+1828.966170769" watchObservedRunningTime="2026-10-30 21:30:46.198631958 +0000 UTC m=+1828.973031897"
Oct 30 21:30:46.244434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:46.244394    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:46.484576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:46.484544    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"]
Oct 30 21:30:46.484885 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:46.484816    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04c3b5cf_40eb_4ec7_8bee_050f87d2fc46.slice/crio-a4e435bdc62d49c22cd256d1aecb2f6723fa89da72ddc400abdb232a1d596144 WatchSource:0}: Error finding container a4e435bdc62d49c22cd256d1aecb2f6723fa89da72ddc400abdb232a1d596144: Status 404 returned error can't find the container with id a4e435bdc62d49c22cd256d1aecb2f6723fa89da72ddc400abdb232a1d596144
Oct 30 21:30:47.117298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.117256    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:47.123708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.123679    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:47.159364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.159325    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" event={"ID":"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46","Type":"ContainerStarted","Data":"11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c"}
Oct 30 21:30:47.159771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.159375    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" event={"ID":"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46","Type":"ContainerStarted","Data":"a4e435bdc62d49c22cd256d1aecb2f6723fa89da72ddc400abdb232a1d596144"}
Oct 30 21:30:47.159771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.159626    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:47.160882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.160863    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:30:47.265184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.265111    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" podStartSLOduration=2.265092319 podStartE2EDuration="2.265092319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:47.260558485 +0000 UTC m=+1830.034958426" watchObservedRunningTime="2026-10-30 21:30:47.265092319 +0000 UTC m=+1830.039492259"
Oct 30 21:30:47.827982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.827949    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:47.944451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:47.944421    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.295701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.295613    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"]
Oct 30 21:30:48.300779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.300756    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.328985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.328958    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"]
Oct 30 21:30:48.424374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424334    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.424550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424389    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.424550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424469    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.424698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424671    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.424739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424725    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.424795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424760    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.425000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.424983    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgpb\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.425151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.425132    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.425352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.425331    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527152    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527289    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527349    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527398    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527453    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527554    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527579    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527611    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527732    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgpb\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527781    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.527987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.527964    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.528092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.528071    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.528195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.528175    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.528447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.528430    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.530300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.530276    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.530398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.530341    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.546965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.546906    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.548767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.548744    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgpb\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb\") pod \"e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") " pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.619496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.619463    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:48.777313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.777266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.974875 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:48.974799    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13f1e31_b3d1_4707_bccf_f0fc82481748.slice/crio-f39a765d43624e8288e80765475c649c3152baf9477e56b126b95e9dbdc4189d WatchSource:0}: Error finding container f39a765d43624e8288e80765475c649c3152baf9477e56b126b95e9dbdc4189d: Status 404 returned error can't find the container with id f39a765d43624e8288e80765475c649c3152baf9477e56b126b95e9dbdc4189d
Oct 30 21:30:48.974984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.974917    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"]
Oct 30 21:30:48.978206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.978174    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:48.978302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.978239    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:48.978302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:48.978269    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:30:49.177144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:49.177096    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" event={"ID":"a13f1e31-b3d1-4707-bccf-f0fc82481748","Type":"ContainerStarted","Data":"a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c"}
Oct 30 21:30:49.177345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:49.177158    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" event={"ID":"a13f1e31-b3d1-4707-bccf-f0fc82481748","Type":"ContainerStarted","Data":"f39a765d43624e8288e80765475c649c3152baf9477e56b126b95e9dbdc4189d"}
Oct 30 21:30:49.204177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:49.204138    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:49.248419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:49.248304    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podStartSLOduration=1.248283684 podStartE2EDuration="1.248283684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:49.244665283 +0000 UTC m=+1832.019065249" watchObservedRunningTime="2026-10-30 21:30:49.248283684 +0000 UTC m=+1832.022683626"
Oct 30 21:30:49.624850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:49.624035    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:49.958503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:49.958395    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:50.504930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:50.504868    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:50.621495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:50.621462    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 21:30:50.621718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:50.621520    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 21:30:50.798909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:50.798752    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:51.620023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:51.619977    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 21:30:51.620235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:51.620059    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 21:30:52.005047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:52.004963    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:52.620582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:52.620545    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 21:30:52.620789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:52.620609    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 21:30:53.620350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:53.620317    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 21:30:53.620807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:53.620378    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 21:30:53.644005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:53.643973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:54.597678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:54.597642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:54.620795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:54.620761    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 21:30:54.621288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:54.620814    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 21:30:54.733031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:54.732994    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:55.288316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.288270    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:55.626545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.626505    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:55.627081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.627025    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:55.628009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.627990    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:30:55.828163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.828132    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:55.842280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.842243    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:55.938088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.938000    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"]
Oct 30 21:30:55.944570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.944541    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:55.963983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.963949    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b2628a\"/\"kube-root-ca.crt\""
Oct 30 21:30:55.963983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.963959    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b2628a\"/\"openshift-service-ca.crt\""
Oct 30 21:30:55.967288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.967264    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b2628a\"/\"default-dockercfg-9584h\""
Oct 30 21:30:55.967448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.967386    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b2628a\"/\"test-model-b2628a-kserve-self-signed-certs\""
Oct 30 21:30:55.969717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:55.969687    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"]
Oct 30 21:30:56.108648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.108395    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.109145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.109098    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.109365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.109347    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.109656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.109636    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.109934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.109914    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lw5q\" (UniqueName: \"kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.110643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.110307    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.159005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.158967    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:56.213008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.212912    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.212962    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lw5q\" (UniqueName: \"kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213206    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213248    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213341    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213409    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213641    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213734    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.213994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.213970    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.215666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.215643    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.216026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.216008    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.232703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.232673    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lw5q\" (UniqueName: \"kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q\") pod \"test-model-b2628a-kserve-76c49f4f-g2xrj\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") " pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.264791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.264755    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:56.538944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.526082    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:30:56.591806 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:30:56.590002    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:30:56.805036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:56.805002    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"]
Oct 30 21:30:56.805368 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:30:56.805129    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda61de4a5_7c25_4ca5_9f38_ed1c4fcc0235.slice/crio-a341b84092201086ebb6153a5e72c29a2b98d100b198e06187a7a886f7cbd9c3 WatchSource:0}: Error finding container a341b84092201086ebb6153a5e72c29a2b98d100b198e06187a7a886f7cbd9c3: Status 404 returned error can't find the container with id a341b84092201086ebb6153a5e72c29a2b98d100b198e06187a7a886f7cbd9c3
Oct 30 21:30:57.386417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:57.386387    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:57.546614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:57.546400    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" event={"ID":"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235","Type":"ContainerStarted","Data":"113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f"}
Oct 30 21:30:57.546614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:57.546453    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" event={"ID":"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235","Type":"ContainerStarted","Data":"a341b84092201086ebb6153a5e72c29a2b98d100b198e06187a7a886f7cbd9c3"}
Oct 30 21:30:57.587763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:57.587708    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" podStartSLOduration=2.587693908 podStartE2EDuration="2.587693908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:57.585747331 +0000 UTC m=+1840.360147269" watchObservedRunningTime="2026-10-30 21:30:57.587693908 +0000 UTC m=+1840.362093847"
Oct 30 21:30:58.264953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:58.264922    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:30:58.823386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:30:58.823346    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:00.352139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:00.352104    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.826192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:01.826157    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:03.206643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:03.206607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:04.657982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:04.657944    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:06.200443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:06.200394    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:06.278862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:06.272415    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:31:06.357151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:06.357123    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:31:06.634031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:06.634000    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:31:07.717250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:07.717209    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.869061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:07.869023    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:09.233195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:09.233165    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.812003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:10.811973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:12.304685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:12.304648    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:13.826171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:13.826137    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.365171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:14.365132    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.502135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:14.502092    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:15.292938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:15.292902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:16.767841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:16.767796    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:18.319261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:18.319219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:19.791010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:19.790912    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:21.242077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:21.242042    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:21.547537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:21.547447    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:22.846617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:22.846581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.342854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:24.336014    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:31:25.138538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:25.138498    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:26.578053 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:31:26.578004    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:31:26.829452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:26.829370    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.347707 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:28.347667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:29.903993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:29.903957    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:31.355579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:31.355542    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:33.002956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:33.002909    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.124752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:34.124718    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.247008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:34.246975    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.610998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:34.610962    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.762485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:34.762445    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.924060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:34.923969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:36.278451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:36.278408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.832457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:37.832426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:39.333692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:39.333658    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:40.915625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:40.915591    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:42.419630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:42.419592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:44.041013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:44.040979    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:45.626362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:45.626320    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:47.188871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:47.184320    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.682215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:48.682174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:50.199326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:50.199288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:51.738108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:51.738038    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:52.254701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:52.254661    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:53.239734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:53.239692    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:54.831006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:54.830961    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:56.262403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:56.262371    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:56.576723 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:31:56.576680    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:31:57.770685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:57.770644    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.310457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:31:59.310421    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:00.703504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:00.703455    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.534997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:02.534957    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:04.785672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:04.785633    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:06.407918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:06.407878    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:07.991333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:07.991293    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:08.126226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:08.126193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:09.699551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:09.699508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:11.199687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:11.199646    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:12.652709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:12.652668    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.385513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.385369    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.675470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.675432    4549 generic.go:356] "Generic (PLEG): container finished" podID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerID="6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d" exitCode=0
Oct 30 21:32:14.675679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.675508    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" event={"ID":"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe","Type":"ContainerDied","Data":"6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"}
Oct 30 21:32:14.676067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.676049    4549 scope.go:122] "RemoveContainer" containerID="6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"
Oct 30 21:32:14.678010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.677980    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:32:14.678076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.678057    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:32:14.678114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:14.678094    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:32:15.689158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:15.689116    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" event={"ID":"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe","Type":"ContainerStarted","Data":"a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855"}
Oct 30 21:32:15.924898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:15.922571    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.117232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:16.117197    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:32:16.118593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:16.118560    4549 patch_prober.go:29] interesting pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 21:32:16.118767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:16.118634    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 21:32:17.117208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:17.117169    4549 patch_prober.go:29] interesting pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 21:32:17.117676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:17.117228    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 21:32:17.160219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:17.160175    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:32:17.723494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:17.723457    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.117772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:18.117733    4549 patch_prober.go:29] interesting pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 21:32:18.118257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:18.117802    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 21:32:19.117726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.117693    4549 patch_prober.go:29] interesting pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 21:32:19.117960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.117764    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 21:32:19.209758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.209714    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"]
Oct 30 21:32:19.210030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.210003    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy" containerID="cri-o://e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf" gracePeriod=30
Oct 30 21:32:19.288896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.280483    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:19.510301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.510202    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"]
Oct 30 21:32:19.510969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:19.510935    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" containerID="cri-o://c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042" gracePeriod=30
Oct 30 21:32:20.117653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:20.117618    4549 patch_prober.go:29] interesting pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 21:32:20.117855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:20.117679    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 21:32:20.809818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:20.809785    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.116897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:21.116860    4549 patch_prober.go:29] interesting pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 21:32:21.117100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:21.116930    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 21:32:22.122761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:22.122731    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:32:22.124089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:22.124068    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:32:22.342079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:22.342043    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:23.623095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:23.622991    4549 patch_prober.go:29] interesting pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:32:23.623844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:23.623793    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:32:23.824901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:23.819446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.513168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.513144    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:32:24.630667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.630624    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.630755    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.630923    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.630985    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.631048    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.631106    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.631234    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.631336    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.631530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.631425    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert\") pod \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\" (UID: \"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83\") "
Oct 30 21:32:24.634086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.634054    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.634278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.634260    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.634436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.634332    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.634436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.634379    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.634654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.634630    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:24.635742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.635710    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:24.636704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.636652    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:24.638301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.638256    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "kube-api-access-94dld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:24.640966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.640938    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy" pod "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" (UID: "46e1ff6d-edaf-46f9-9653-4dc6bb3eab83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.734497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734461    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734500    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734514    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734526    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734539    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94dld\" (UniqueName: \"kubernetes.io/projected/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-kube-api-access-94dld\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734555    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734568    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734581    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.734805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.734594    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.807359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.807332    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:32:24.882339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.882280    4549 generic.go:356] "Generic (PLEG): container finished" podID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerID="e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf" exitCode=0
Oct 30 21:32:24.882631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.882375    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" event={"ID":"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83","Type":"ContainerDied","Data":"e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf"}
Oct 30 21:32:24.882631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.882447    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq" event={"ID":"46e1ff6d-edaf-46f9-9653-4dc6bb3eab83","Type":"ContainerDied","Data":"ae298260cb91cbccb5793719a4cd1cb0d5a8feb4b9a68fda482098d34b99ca45"}
Oct 30 21:32:24.882631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.882479    4549 scope.go:122] "RemoveContainer" containerID="e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf"
Oct 30 21:32:24.882923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.882644    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"
Oct 30 21:32:24.889567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.889505    4549 generic.go:356] "Generic (PLEG): container finished" podID="d7ffaa97-07b5-4419-b324-f263db44366c" containerID="c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042" exitCode=0
Oct 30 21:32:24.889693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.889572    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"
Oct 30 21:32:24.889693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.889621    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" event={"ID":"d7ffaa97-07b5-4419-b324-f263db44366c","Type":"ContainerDied","Data":"c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042"}
Oct 30 21:32:24.889843 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.889692    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg" event={"ID":"d7ffaa97-07b5-4419-b324-f263db44366c","Type":"ContainerDied","Data":"516c319fbaedae1d0a558c38a69ff5a1fd67a4a54119fee73c252b7685c9f706"}
Oct 30 21:32:24.911683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.911655    4549 scope.go:122] "RemoveContainer" containerID="0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"
Oct 30 21:32:24.928253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.928221    4549 scope.go:122] "RemoveContainer" containerID="e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf"
Oct 30 21:32:24.928672 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:32:24.928643    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf\": container with ID starting with e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf not found: ID does not exist" containerID="e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf"
Oct 30 21:32:24.928774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.928685    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf"} err="failed to get container status \"e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf\": rpc error: code = NotFound desc = could not find container \"e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf\": container with ID starting with e9d354139e22e8078538258fed7ddfcf95e79913834b4fc2dc725d4b21e66cbf not found: ID does not exist"
Oct 30 21:32:24.928774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.928708    4549 scope.go:122] "RemoveContainer" containerID="0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"
Oct 30 21:32:24.929997 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:32:24.929968    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191\": container with ID starting with 0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191 not found: ID does not exist" containerID="0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"
Oct 30 21:32:24.930112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.930004    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191"} err="failed to get container status \"0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191\": rpc error: code = NotFound desc = could not find container \"0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191\": container with ID starting with 0ca6c56ef505c04376167b78a4bd2f05339259e045ddedf1a37b0db61f869191 not found: ID does not exist"
Oct 30 21:32:24.930112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.930023    4549 scope.go:122] "RemoveContainer" containerID="c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042"
Oct 30 21:32:24.936970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.936943    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937011    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937041    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937202    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937331    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937438    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937495    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937543    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.937671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937561    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937627    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz\") pod \"d7ffaa97-07b5-4419-b324-f263db44366c\" (UID: \"d7ffaa97-07b5-4419-b324-f263db44366c\") "
Oct 30 21:32:24.937857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.937762    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.938268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.938245    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.938526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.938503    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:24.939121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.939098    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.940248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.940221    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.940364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.940252    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.940364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.940266    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ffaa97-07b5-4419-b324-f263db44366c-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.940364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.940279    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.940364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.940293    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:24.944302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.944234    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:24.944624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.944602    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:24.944803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.944636    4549 scope.go:122] "RemoveContainer" containerID="c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042"
Oct 30 21:32:24.945044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.944992    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:24.945250 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:32:24.945221    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042\": container with ID starting with c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042 not found: ID does not exist" containerID="c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042"
Oct 30 21:32:24.945322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.945262    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042"} err="failed to get container status \"c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042\": rpc error: code = NotFound desc = could not find container \"c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042\": container with ID starting with c56517c9813b6a558ae3eb9e5dd9ee6ec5bc20e254f9b31bb67c84d0fbe6b042 not found: ID does not exist"
Oct 30 21:32:24.945904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.945884    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz" pod "d7ffaa97-07b5-4419-b324-f263db44366c" (UID: "d7ffaa97-07b5-4419-b324-f263db44366c"). InnerVolumeSpecName "kube-api-access-289xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:24.957517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.957490    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"]
Oct 30 21:32:24.963208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:24.963188    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq"]
Oct 30 21:32:25.041835 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.041789    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:25.041835 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.041818    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ffaa97-07b5-4419-b324-f263db44366c-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:25.042053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.041844    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-289xz\" (UniqueName: \"kubernetes.io/projected/d7ffaa97-07b5-4419-b324-f263db44366c-kube-api-access-289xz\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:25.042053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.041859    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ffaa97-07b5-4419-b324-f263db44366c-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:32:25.244743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.244720    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"]
Oct 30 21:32:25.257780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.257753    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg"]
Oct 30 21:32:25.347473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.347437    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" path="/var/lib/kubelet/pods/46e1ff6d-edaf-46f9-9653-4dc6bb3eab83/volumes"
Oct 30 21:32:25.348135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.348117    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" path="/var/lib/kubelet/pods/d7ffaa97-07b5-4419-b324-f263db44366c/volumes"
Oct 30 21:32:25.702447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.702412    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.909228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:25.909198    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:26.577846 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:32:26.577792    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:32:27.220430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.220366    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:27.288120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.288088    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"]
Oct 30 21:32:27.289162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289133    4549 cpu_manager.go:414] "containerMap: removing container" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy"
Oct 30 21:32:27.289162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289162    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy"
Oct 30 21:32:27.289367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289177    4549 cpu_manager.go:414] "containerMap: removing container" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy"
Oct 30 21:32:27.289367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289185    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy"
Oct 30 21:32:27.289443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289410    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy"
Oct 30 21:32:27.289443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289425    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7ffaa97-07b5-4419-b324-f263db44366c" containerName="istio-proxy"
Oct 30 21:32:27.289513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.289445    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy"
Oct 30 21:32:27.294889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.294821    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.314372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.314338    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"]
Oct 30 21:32:27.336053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.335212    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-67f8e0\"/\"default-dockercfg-swzv8\""
Oct 30 21:32:27.336053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.335538    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-67f8e0\"/\"test-model-67f8e0-kserve-self-signed-certs\""
Oct 30 21:32:27.339860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.336974    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-67f8e0\"/\"openshift-service-ca.crt\""
Oct 30 21:32:27.339860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.337258    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-67f8e0\"/\"kube-root-ca.crt\""
Oct 30 21:32:27.383129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.383092    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:27.395761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.395725    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.395955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.395877    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.396067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.396043    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzrgp\" (UniqueName: \"kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.396200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.396180    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.396370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.396344    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.396453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.396420    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.497761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.497657    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.497976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.497874    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.497976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.497935    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.498101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.498055    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.498157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.498133    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.498307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.498267    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzrgp\" (UniqueName: \"kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.498446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.498269    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.498513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.498455    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.498571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.498535    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.500394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.500366    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.500565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.500546    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.518687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.518655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzrgp\" (UniqueName: \"kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp\") pod \"test-model-67f8e0-kserve-57c44b8c4c-ndb8m\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") " pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:27.628964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:27.615166    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:28.086062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:28.086027    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"]
Oct 30 21:32:28.086791 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:32:28.086762    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a15e9fd_6ae8_4f1e_9b5c_cc16cbc77cc0.slice/crio-930225d4ca7b417b2b2d9820ae14afd308a9243ab7ff224def73844c2287f89c WatchSource:0}: Error finding container 930225d4ca7b417b2b2d9820ae14afd308a9243ab7ff224def73844c2287f89c: Status 404 returned error can't find the container with id 930225d4ca7b417b2b2d9820ae14afd308a9243ab7ff224def73844c2287f89c
Oct 30 21:32:28.813331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:28.813291    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:28.936592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:28.936559    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" event={"ID":"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0","Type":"ContainerStarted","Data":"db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac"}
Oct 30 21:32:28.936810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:28.936609    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" event={"ID":"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0","Type":"ContainerStarted","Data":"930225d4ca7b417b2b2d9820ae14afd308a9243ab7ff224def73844c2287f89c"}
Oct 30 21:32:28.981764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:28.981688    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" podStartSLOduration=1.981668448 podStartE2EDuration="1.981668448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:28.977033489 +0000 UTC m=+1931.751433428" watchObservedRunningTime="2026-10-30 21:32:28.981668448 +0000 UTC m=+1931.756068387"
Oct 30 21:32:29.616368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:29.616330    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:30.425925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:30.423318    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:32.009891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:32.000497    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.479186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:33.479140    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.171023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:35.170984    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.631505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:36.630286    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:37.615819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:37.615785    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:37.629317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:37.629288    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:38.046916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:38.046768    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.073543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:38.073507    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:32:39.725749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:39.725711    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.372451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:41.372413    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:42.228137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:42.228096    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:42.419952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:42.419909    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:42.835793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:42.835726    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.432699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:44.432666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:46.131857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:46.121677    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:47.319445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:47.319410    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"]
Oct 30 21:32:47.382033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:47.382005    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:47.583734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:47.583644    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.346721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:49.346684    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:50.804206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:50.804168    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:52.226611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:52.226573    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:53.831320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:53.831285    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:55.516086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:55.516052    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:56.578311 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:32:56.578266    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:32:57.105390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:57.105352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:58.608761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:32:58.608721    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:00.098889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:00.098815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:01.568629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:01.568589    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:02.350137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.350090    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerName="main" containerID="cri-o://fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095" gracePeriod=45
Oct 30 21:33:02.662381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.662353    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:33:02.760587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.760533    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f\" (UniqueName: \"kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f\") pod \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") "
Oct 30 21:33:02.760813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.760654    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home\") pod \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") "
Oct 30 21:33:02.760813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.760689    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs\" (UniqueName: \"kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs\") pod \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") "
Oct 30 21:33:02.760962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.760856    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir\") pod \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") "
Oct 30 21:33:02.761145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.761124    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache\") pod \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") "
Oct 30 21:33:02.761253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.761164    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm\") pod \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\" (UID: \"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a\") "
Oct 30 21:33:02.761253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.761178    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir" pod "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" (UID: "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:02.761413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.761388    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache" pod "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" (UID: "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:02.761539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.761517    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home" pod "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" (UID: "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:02.762270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.762245    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:02.762402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.762275    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:02.762402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.762292    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:02.763453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.763430    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs" pod "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" (UID: "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:02.763540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.763477    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f" pod "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" (UID: "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a"). InnerVolumeSpecName "kube-api-access-2qh4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:02.764017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.764001    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm" pod "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" (UID: "7bbb76b0-9f3a-4686-b111-1c7d2f27a55a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:02.864282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.864241    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:02.864282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.864275    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qh4f\" (UniqueName: \"kubernetes.io/projected/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-kube-api-access-2qh4f\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:02.864282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:02.864290    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:03.016440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.016341    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:03.397946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.397913    4549 generic.go:356] "Generic (PLEG): container finished" podID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerID="fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095" exitCode=0
Oct 30 21:33:03.398120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.397972    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"
Oct 30 21:33:03.398120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.397975    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" event={"ID":"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a","Type":"ContainerDied","Data":"fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095"}
Oct 30 21:33:03.398120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.398077    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p" event={"ID":"7bbb76b0-9f3a-4686-b111-1c7d2f27a55a","Type":"ContainerDied","Data":"1fc69c37a6430f0ca48bf28ef09bfadcb7d5ddc45d9fac45d4fac5650a002baf"}
Oct 30 21:33:03.398120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.398093    4549 scope.go:122] "RemoveContainer" containerID="fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095"
Oct 30 21:33:03.409320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.409294    4549 scope.go:122] "RemoveContainer" containerID="fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095"
Oct 30 21:33:03.409667 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:33:03.409644    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095\": container with ID starting with fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095 not found: ID does not exist" containerID="fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095"
Oct 30 21:33:03.409753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.409675    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095"} err="failed to get container status \"fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095\": rpc error: code = NotFound desc = could not find container \"fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095\": container with ID starting with fa85541d64c1786e68692666fb0ff006bae7ff6ed13e9718b4b06539e23e1095 not found: ID does not exist"
Oct 30 21:33:03.451036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.451003    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"]
Oct 30 21:33:03.463169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:03.463132    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c31999/test-model-c31999-kserve-5748d95599-x7s6p"]
Oct 30 21:33:04.564861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.562116    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:04.781496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.781460    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"]
Oct 30 21:33:04.782552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.782527    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerName="main"
Oct 30 21:33:04.782552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.782549    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerName="main"
Oct 30 21:33:04.782755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.782583    4549 cpu_manager.go:414] "containerMap: removing container" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy"
Oct 30 21:33:04.782755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.782588    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46e1ff6d-edaf-46f9-9653-4dc6bb3eab83" containerName="istio-proxy"
Oct 30 21:33:04.782755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.782748    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" containerName="main"
Oct 30 21:33:04.788406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.788386    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:04.809130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.809098    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"]
Oct 30 21:33:04.813002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.812970    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f23eae\"/\"openshift-service-ca.crt\""
Oct 30 21:33:04.813194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.813151    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f23eae\"/\"test-model-f23eae-kserve-self-signed-certs\""
Oct 30 21:33:04.814725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.814704    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f23eae\"/\"default-dockercfg-lj5wk\""
Oct 30 21:33:04.814953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.814816    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f23eae\"/\"kube-root-ca.crt\""
Oct 30 21:33:04.888416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.888378    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:04.909956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.909922    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm8cz\" (UniqueName: \"kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:04.909956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.909967    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:04.910179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.909990    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:04.910217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.910171    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:04.910251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.910229    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:04.910469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:04.910451    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012050    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012145    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm8cz\" (UniqueName: \"kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012176    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012193    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012298    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012368    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012486    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012588    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.012805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.012785    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.014910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.014881    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.015204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.015182    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.088912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.084784    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm8cz\" (UniqueName: \"kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz\") pod \"test-model-f23eae-kserve-78bc8dbb7-sjhlx\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") " pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.115897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.111212    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:05.352642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.352556    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bbb76b0-9f3a-4686-b111-1c7d2f27a55a" path="/var/lib/kubelet/pods/7bbb76b0-9f3a-4686-b111-1c7d2f27a55a/volumes"
Oct 30 21:33:05.552198 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:33:05.552168    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde2fc55a_f5dd_481a_99f4_a6d04c9c9cf4.slice/crio-1a624c5fba15092790cb7b242218dac33a253d2b79aa8b44b547e4fd1d99a902 WatchSource:0}: Error finding container 1a624c5fba15092790cb7b242218dac33a253d2b79aa8b44b547e4fd1d99a902: Status 404 returned error can't find the container with id 1a624c5fba15092790cb7b242218dac33a253d2b79aa8b44b547e4fd1d99a902
Oct 30 21:33:05.552487 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:05.552461    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"]
Oct 30 21:33:06.150105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:06.150069    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:06.425290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:06.425195    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx" event={"ID":"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4","Type":"ContainerStarted","Data":"dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199"}
Oct 30 21:33:06.425290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:06.425248    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx" event={"ID":"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4","Type":"ContainerStarted","Data":"1a624c5fba15092790cb7b242218dac33a253d2b79aa8b44b547e4fd1d99a902"}
Oct 30 21:33:06.464713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:06.464660    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx" podStartSLOduration=2.464646661 podStartE2EDuration="2.464646661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:06.464091981 +0000 UTC m=+1969.238491937" watchObservedRunningTime="2026-10-30 21:33:06.464646661 +0000 UTC m=+1969.239046601"
Oct 30 21:33:07.111659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:07.111625    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:07.702407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:07.702369    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:08.121973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:08.121935    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:09.342854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.336941    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:09.423856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.423087    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:09.444854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.443676    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:33:09.584858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.578947    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:09.584858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.580030    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" podUID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" containerName="main" probeResult="failure" output="Get \"https://10.133.0.168:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:09.607911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.601514    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:09.641855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.641191    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerName="main" probeResult="failure" output="Get \"https://10.133.0.170:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:09.938502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:09.938391    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" podUID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.153:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:33:10.094855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:10.094087    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 21:33:10.707434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:10.707399    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:12.222241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:12.222204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:13.714328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:13.714295    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.111644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:15.111607    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:15.137508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:15.137478    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:15.176004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:15.175929    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.847581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:15.847551    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:33:16.616219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:16.616181    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.231562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:18.231514    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:19.813761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:19.813651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:21.285022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:21.284983    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.848860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:22.848785    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:24.231129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:24.231089    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:25.200987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:25.200950    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"]
Oct 30 21:33:25.276284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:25.276250    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.033709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:26.033669    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.577920 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:33:26.577881    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:33:27.708434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:27.708342    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.212383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:29.212343    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:30.827386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:30.827299    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.273159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:32.273116    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.902187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:33.902129    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.325365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:35.325322    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:36.278445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:36.278404    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:36.421528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:36.421486    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.370717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:37.370649    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:38.841113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:38.841080    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:40.225728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.225661    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" podUID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" containerName="main" containerID="cri-o://6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee" gracePeriod=45
Oct 30 21:33:40.346600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.346544    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:40.598121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.598097    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:33:40.623617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.623585    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir\") pod \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") "
Oct 30 21:33:40.623789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.623733    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr\" (UniqueName: \"kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr\") pod \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") "
Oct 30 21:33:40.623871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.623804    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache\") pod \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") "
Oct 30 21:33:40.623926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.623863    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir" pod "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" (UID: "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:40.624031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624012    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs\") pod \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") "
Oct 30 21:33:40.624128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624074    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache" pod "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" (UID: "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:40.624128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624121    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home\") pod \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") "
Oct 30 21:33:40.624206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624163    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm\") pod \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\" (UID: \"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea\") "
Oct 30 21:33:40.624379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624353    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home" pod "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" (UID: "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:40.624905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624882    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:40.624905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624909    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:40.624905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.624922    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:40.627128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.627073    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs" pod "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" (UID: "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:40.627128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.627089    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm" pod "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" (UID: "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:40.627314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.627296    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr" pod "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" (UID: "e745e0e6-7871-4b4d-8f3b-241d3a83b4ea"). InnerVolumeSpecName "kube-api-access-qvwgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:40.726188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.726152    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:40.726188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.726184    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:40.726401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:40.726197    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvwgr\" (UniqueName: \"kubernetes.io/projected/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea-kube-api-access-qvwgr\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:33:41.124533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.124497    4549 generic.go:356] "Generic (PLEG): container finished" podID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" containerID="6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee" exitCode=0
Oct 30 21:33:41.124764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.124547    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"
Oct 30 21:33:41.124764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.124595    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" event={"ID":"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea","Type":"ContainerDied","Data":"6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee"}
Oct 30 21:33:41.124764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.124649    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs" event={"ID":"e745e0e6-7871-4b4d-8f3b-241d3a83b4ea","Type":"ContainerDied","Data":"6e6bea77eb7f1937b002eb432591da8bfc0ca29bd1d6e22c9ed60f9335f22d7e"}
Oct 30 21:33:41.124764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.124671    4549 scope.go:122] "RemoveContainer" containerID="6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee"
Oct 30 21:33:41.136923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.136900    4549 scope.go:122] "RemoveContainer" containerID="6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee"
Oct 30 21:33:41.139625 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:33:41.139600    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee\": container with ID starting with 6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee not found: ID does not exist" containerID="6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee"
Oct 30 21:33:41.139753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.139633    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee"} err="failed to get container status \"6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee\": rpc error: code = NotFound desc = could not find container \"6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee\": container with ID starting with 6f484063755ab035714c366222f5e9ea6e43a748712a0fec452201696c3a82ee not found: ID does not exist"
Oct 30 21:33:41.177581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.177541    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"]
Oct 30 21:33:41.181944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.181918    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-093daa/test-model-093daa-kserve-744f4978f5-4xzjs"]
Oct 30 21:33:41.347816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.347781    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" path="/var/lib/kubelet/pods/e745e0e6-7871-4b4d-8f3b-241d3a83b4ea/volumes"
Oct 30 21:33:41.828910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:41.828878    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:43.292636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:43.292596    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:46.059506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:46.059458    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.741984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:47.741949    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:49.403065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:49.403030    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.034054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:51.034013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.627998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:52.627959    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:54.115533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:54.115503    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:55.616260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:55.616210    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.577013 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:33:56.576969    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:33:57.317865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:57.317806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.827890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:33:58.827772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.354353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:00.354316    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:02.107970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:02.107931    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:03.614359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:03.614317    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.038325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:05.038288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:06.522407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:06.522373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.351704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:07.351671    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"]
Oct 30 21:34:07.351897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:07.351880    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy" containerID="cri-o://9916b0cffac09d10598f6e215b7ddbae00b2247008529d869b69a1cd621ba377" gracePeriod=30
Oct 30 21:34:07.801191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:07.801106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.394221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:08.394175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.561711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:08.561676    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.694125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:08.694022    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.825660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:08.825607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:09.255412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:09.255376    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.339193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:10.339161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:11.901855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:11.895592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:12.605294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.605255    4549 generic.go:356] "Generic (PLEG): container finished" podID="123cd94c-22ac-4613-92c7-be45db78256c" containerID="9916b0cffac09d10598f6e215b7ddbae00b2247008529d869b69a1cd621ba377" exitCode=0
Oct 30 21:34:12.605453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.605331    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" event={"ID":"123cd94c-22ac-4613-92c7-be45db78256c","Type":"ContainerDied","Data":"9916b0cffac09d10598f6e215b7ddbae00b2247008529d869b69a1cd621ba377"}
Oct 30 21:34:12.670482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.670452    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:34:12.735715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.735684    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.735940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.735787    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.735940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.735896    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.735940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.735935    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.736142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736038    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.736142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736094    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.736142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736138    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.736310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736189    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.736377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736316    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert\") pod \"123cd94c-22ac-4613-92c7-be45db78256c\" (UID: \"123cd94c-22ac-4613-92c7-be45db78256c\") "
Oct 30 21:34:12.736439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736381    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:12.736601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736581    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:12.736992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.736966    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:12.737516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.737492    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.737632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.737523    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.737777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.737754    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:12.737905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.737875    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:12.742335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.742305    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:12.742586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.742546    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "kube-api-access-l2x62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:12.742883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.742777    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:12.746477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.746441    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo" pod "123cd94c-22ac-4613-92c7-be45db78256c" (UID: "123cd94c-22ac-4613-92c7-be45db78256c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:34:12.849648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.849135    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.849925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.849178    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.850012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.849944    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.850012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.849962    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2x62\" (UniqueName: \"kubernetes.io/projected/123cd94c-22ac-4613-92c7-be45db78256c-kube-api-access-l2x62\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.850012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.849977    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/123cd94c-22ac-4613-92c7-be45db78256c-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.850012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.849992    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/123cd94c-22ac-4613-92c7-be45db78256c-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:12.850012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:12.850003    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/123cd94c-22ac-4613-92c7-be45db78256c-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:13.370070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:13.370035    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:13.614690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:13.614602    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"
Oct 30 21:34:13.614690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:13.614633    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29" event={"ID":"123cd94c-22ac-4613-92c7-be45db78256c","Type":"ContainerDied","Data":"bc23e77ab49ce2fe69534f5842b4ca992586fcaf429189e87f88298e45223886"}
Oct 30 21:34:13.614999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:13.614700    4549 scope.go:122] "RemoveContainer" containerID="9916b0cffac09d10598f6e215b7ddbae00b2247008529d869b69a1cd621ba377"
Oct 30 21:34:13.680616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:13.680581    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"]
Oct 30 21:34:13.692268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:13.692238    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29"]
Oct 30 21:34:14.936707 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:14.936670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.348257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:15.348220    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="123cd94c-22ac-4613-92c7-be45db78256c" path="/var/lib/kubelet/pods/123cd94c-22ac-4613-92c7-be45db78256c/volumes"
Oct 30 21:34:15.599534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:15.599433    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.726020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:15.725988    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.911848 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:15.911729    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"]
Oct 30 21:34:15.912042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:15.911983    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" containerID="cri-o://06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0" gracePeriod=30
Oct 30 21:34:15.986930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:15.986888    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:16.688773 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:16.688733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:18.154622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:18.154578    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:18.284937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:18.284902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:18.337120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:18.337087    4549 patch_prober.go:29] interesting pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:34:18.337335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:18.337160    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:34:18.419513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:18.419423    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:19.839339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:19.839305    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:21.242671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.242644    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:34:21.321931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.321895    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.386939    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387073    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387279    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387359    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387394    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387453    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387535    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387585    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.400624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.387656    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data\") pod \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\" (UID: \"2be14cc1-2729-4b2a-a37f-b01dd0b27a84\") "
Oct 30 21:34:21.402123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.402077    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:21.404359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.404305    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:21.404749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.404681    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:21.404962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.404938    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:21.407042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.405461    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:21.417869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.413790    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:21.438863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.438769    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "kube-api-access-rnflj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:21.440965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.440932    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:21.442316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.442292    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo" pod "2be14cc1-2729-4b2a-a37f-b01dd0b27a84" (UID: "2be14cc1-2729-4b2a-a37f-b01dd0b27a84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:34:21.490225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490191    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490222    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490234    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490246    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490257    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490270    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnflj\" (UniqueName: \"kubernetes.io/projected/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-kube-api-access-rnflj\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490281    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490293    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.490491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.490304    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be14cc1-2729-4b2a-a37f-b01dd0b27a84-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:21.705446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.705345    4549 generic.go:356] "Generic (PLEG): container finished" podID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerID="06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0" exitCode=0
Oct 30 21:34:21.705446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.705404    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" event={"ID":"2be14cc1-2729-4b2a-a37f-b01dd0b27a84","Type":"ContainerDied","Data":"06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0"}
Oct 30 21:34:21.705446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.705417    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"
Oct 30 21:34:21.705708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.705457    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94" event={"ID":"2be14cc1-2729-4b2a-a37f-b01dd0b27a84","Type":"ContainerDied","Data":"84e608f651b1142c8ad64ff80e34c984108810d3704e332119351dc716e1d245"}
Oct 30 21:34:21.705708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.705483    4549 scope.go:122] "RemoveContainer" containerID="06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0"
Oct 30 21:34:21.721337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.720770    4549 scope.go:122] "RemoveContainer" containerID="e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"
Oct 30 21:34:21.746785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.746640    4549 scope.go:122] "RemoveContainer" containerID="06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0"
Oct 30 21:34:21.749891 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:34:21.749329    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0\": container with ID starting with 06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0 not found: ID does not exist" containerID="06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0"
Oct 30 21:34:21.749891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.749378    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0"} err="failed to get container status \"06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0\": rpc error: code = NotFound desc = could not find container \"06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0\": container with ID starting with 06d3f6bb239e6c00ea97d594ac55384003159a497c97a608d7a1975397588ae0 not found: ID does not exist"
Oct 30 21:34:21.749891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.749408    4549 scope.go:122] "RemoveContainer" containerID="e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"
Oct 30 21:34:21.774779 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:34:21.754925    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3\": container with ID starting with e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3 not found: ID does not exist" containerID="e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"
Oct 30 21:34:21.774779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.754980    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3"} err="failed to get container status \"e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3\": rpc error: code = NotFound desc = could not find container \"e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3\": container with ID starting with e7523b893b1108bbbf5e734f4d70a24b96bbbfeec5386121469aa1abedfa73b3 not found: ID does not exist"
Oct 30 21:34:21.836852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.828892    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"]
Oct 30 21:34:21.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:21.848176    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94"]
Oct 30 21:34:23.026617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:23.026581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:23.348491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:23.348460    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" path="/var/lib/kubelet/pods/2be14cc1-2729-4b2a-a37f-b01dd0b27a84/volumes"
Oct 30 21:34:24.304360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:24.304325    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:24.617411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:24.617366    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.062122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:26.062087    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.574936 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:34:26.574901    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:34:26.964084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:26.963986    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"]
Oct 30 21:34:26.996082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:26.996043    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"]
Oct 30 21:34:27.125510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:27.125470    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:27.266910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:27.262384    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:27.766854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:27.766730    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.547501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:29.547469    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:31.209149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:31.209118    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:31.965310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:31.965254    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerName="payload-processing" containerID="cri-o://5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab" gracePeriod=25
Oct 30 21:34:31.997290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:31.997246    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerName="payload-pre-processing" containerID="cri-o://7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898" gracePeriod=25
Oct 30 21:34:32.692129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:32.692089    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:34.353447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:34.353410    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:35.779969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:35.779911    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:36.377079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:36.377041    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:36.389393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:36.389361    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:37.202648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:37.202605    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:38.620153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:38.620120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:40.006304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:40.006246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:41.514468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:41.514431    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:43.032513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:43.032474    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:44.477803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:44.477767    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:45.985695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:45.985660    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:46.376403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:46.376360    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:46.389493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:46.389456    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:47.546601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:47.546562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:49.041404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:49.041370    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:50.603967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:50.603928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:51.988672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:51.988630    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:52.327696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.327668    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:34:52.375153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.375124    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:34:52.492995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.492898    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume\") pod \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") "
Oct 30 21:34:52.493168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.493073    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77\" (UniqueName: \"kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77\") pod \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\" (UID: \"87fda6f5-520b-4855-a3d6-de68f43bf1c8\") "
Oct 30 21:34:52.493222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.493168    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5\" (UniqueName: \"kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5\") pod \"0f2dc276-80c4-48f1-9a81-2e5197378916\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") "
Oct 30 21:34:52.493287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.493260    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume\") pod \"0f2dc276-80c4-48f1-9a81-2e5197378916\" (UID: \"0f2dc276-80c4-48f1-9a81-2e5197378916\") "
Oct 30 21:34:52.493360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.493333    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume" pod "87fda6f5-520b-4855-a3d6-de68f43bf1c8" (UID: "87fda6f5-520b-4855-a3d6-de68f43bf1c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:52.493701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.493632    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume" pod "0f2dc276-80c4-48f1-9a81-2e5197378916" (UID: "0f2dc276-80c4-48f1-9a81-2e5197378916"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:52.494285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.494265    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2dc276-80c4-48f1-9a81-2e5197378916-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:52.494285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.494286    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87fda6f5-520b-4855-a3d6-de68f43bf1c8-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:52.495643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.495617    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5" pod "0f2dc276-80c4-48f1-9a81-2e5197378916" (UID: "0f2dc276-80c4-48f1-9a81-2e5197378916"). InnerVolumeSpecName "kube-api-access-4z2p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:52.495938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.495917    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77" pod "87fda6f5-520b-4855-a3d6-de68f43bf1c8" (UID: "87fda6f5-520b-4855-a3d6-de68f43bf1c8"). InnerVolumeSpecName "kube-api-access-gvv77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:52.596089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.596053    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvv77\" (UniqueName: \"kubernetes.io/projected/87fda6f5-520b-4855-a3d6-de68f43bf1c8-kube-api-access-gvv77\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:52.596089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:52.596083    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z2p5\" (UniqueName: \"kubernetes.io/projected/0f2dc276-80c4-48f1-9a81-2e5197378916-kube-api-access-4z2p5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:34:53.078819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.078774    4549 generic.go:356] "Generic (PLEG): container finished" podID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerID="5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab" exitCode=0
Oct 30 21:34:53.079375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.078821    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" event={"ID":"0f2dc276-80c4-48f1-9a81-2e5197378916","Type":"ContainerDied","Data":"5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab"}
Oct 30 21:34:53.079375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.078884    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"
Oct 30 21:34:53.079375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.078935    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7" event={"ID":"0f2dc276-80c4-48f1-9a81-2e5197378916","Type":"ContainerDied","Data":"240b52799c338e3d8052a7655584cb1b761506999832fc4d19909d10df363633"}
Oct 30 21:34:53.079375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.078969    4549 scope.go:122] "RemoveContainer" containerID="5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab"
Oct 30 21:34:53.080574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.080549    4549 generic.go:356] "Generic (PLEG): container finished" podID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerID="7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898" exitCode=0
Oct 30 21:34:53.080810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.080589    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"
Oct 30 21:34:53.080810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.080608    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" event={"ID":"87fda6f5-520b-4855-a3d6-de68f43bf1c8","Type":"ContainerDied","Data":"7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898"}
Oct 30 21:34:53.080810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.080652    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78" event={"ID":"87fda6f5-520b-4855-a3d6-de68f43bf1c8","Type":"ContainerDied","Data":"98ff286478b324b7c68b680011e2b368af73361243e04fa51a8fa1e8f357a9c0"}
Oct 30 21:34:53.091384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.091357    4549 scope.go:122] "RemoveContainer" containerID="5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab"
Oct 30 21:34:53.091705 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:34:53.091685    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab\": container with ID starting with 5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab not found: ID does not exist" containerID="5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab"
Oct 30 21:34:53.091789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.091719    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab"} err="failed to get container status \"5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab\": rpc error: code = NotFound desc = could not find container \"5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab\": container with ID starting with 5d151062d775863e323d35d15112b99661b811179d6b92243ca3bbcb8f5fb6ab not found: ID does not exist"
Oct 30 21:34:53.091789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.091744    4549 scope.go:122] "RemoveContainer" containerID="7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898"
Oct 30 21:34:53.102324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.102305    4549 scope.go:122] "RemoveContainer" containerID="7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898"
Oct 30 21:34:53.102670 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:34:53.102637    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898\": container with ID starting with 7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898 not found: ID does not exist" containerID="7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898"
Oct 30 21:34:53.102726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.102672    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898"} err="failed to get container status \"7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898\": rpc error: code = NotFound desc = could not find container \"7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898\": container with ID starting with 7ae7b4ed5343035ca08548880cf79a028e6111d6329d52ff128f041582ecd898 not found: ID does not exist"
Oct 30 21:34:53.131491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.131463    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"]
Oct 30 21:34:53.143043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.143015    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78"]
Oct 30 21:34:53.176475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.176430    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"]
Oct 30 21:34:53.180692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.180664    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7"]
Oct 30 21:34:53.347200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.347169    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" path="/var/lib/kubelet/pods/0f2dc276-80c4-48f1-9a81-2e5197378916/volumes"
Oct 30 21:34:53.347657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.347643    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" path="/var/lib/kubelet/pods/87fda6f5-520b-4855-a3d6-de68f43bf1c8/volumes"
Oct 30 21:34:53.538037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:53.538006    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:55.007560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:55.007527    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:56.407105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:56.407069    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:56.576060 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:34:56.576022    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:34:57.678545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:57.678509    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:58.198073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:34:58.198030    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:00.059919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:00.059859    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" podUID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.171:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:00.106858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:00.103867    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:35:00.378234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:00.378204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:01.093590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:01.093550    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:01.235565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:01.235529    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:01.359360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:01.359268    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:01.760327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:01.760226    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:02.960377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:02.960340    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:04.434204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:04.434175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:05.890284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:05.890244    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:07.560861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:07.557073    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:07.867166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:07.867128    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:07.976103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:07.976061    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"]
Oct 30 21:35:07.976388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:07.976358    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy" containerID="cri-o://5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac" gracePeriod=30
Oct 30 21:35:08.093854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:08.090355    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:08.505782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:08.505749    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:09.326223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:09.326191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:10.753881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:10.753764    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:12.125908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:12.125874    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:13.270696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.270665    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:35:13.316259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316216    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316405    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316503    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316545    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316614    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316667    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316725    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316767    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.316946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316855    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo\") pod \"22f11dbc-7817-471f-b839-0b277254e60b\" (UID: \"22f11dbc-7817-471f-b839-0b277254e60b\") "
Oct 30 21:35:13.317011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.316932    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:35:13.317301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.317247    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:35:13.317629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.317586    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:13.318042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.318018    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.318159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.318049    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.318159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.318068    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f11dbc-7817-471f-b839-0b277254e60b-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.318396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.318375    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:35:13.318471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.318433    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:35:13.319781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.319754    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:35:13.320489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.320451    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:13.321178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.321087    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:35:13.321527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.321196    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg" pod "22f11dbc-7817-471f-b839-0b277254e60b" (UID: "22f11dbc-7817-471f-b839-0b277254e60b"). InnerVolumeSpecName "kube-api-access-fncmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:13.420582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.420541    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.420582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.420582    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.420878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.420595    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fncmg\" (UniqueName: \"kubernetes.io/projected/22f11dbc-7817-471f-b839-0b277254e60b-kube-api-access-fncmg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.420878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.420611    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.420878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.420623    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f11dbc-7817-471f-b839-0b277254e60b-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.420878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.420635    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f11dbc-7817-471f-b839-0b277254e60b-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:35:13.457925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.457889    4549 generic.go:356] "Generic (PLEG): container finished" podID="22f11dbc-7817-471f-b839-0b277254e60b" containerID="5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac" exitCode=0
Oct 30 21:35:13.458107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.457942    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"
Oct 30 21:35:13.458107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.457963    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" event={"ID":"22f11dbc-7817-471f-b839-0b277254e60b","Type":"ContainerDied","Data":"5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac"}
Oct 30 21:35:13.458107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.458019    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt" event={"ID":"22f11dbc-7817-471f-b839-0b277254e60b","Type":"ContainerDied","Data":"4ddab8f3d7c3c113ad5b85ac4179af853f0cc375f8ced0324fb9a02eb3ec5221"}
Oct 30 21:35:13.458107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.458045    4549 scope.go:122] "RemoveContainer" containerID="5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac"
Oct 30 21:35:13.470245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.470226    4549 scope.go:122] "RemoveContainer" containerID="1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"
Oct 30 21:35:13.480904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.480883    4549 scope.go:122] "RemoveContainer" containerID="5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac"
Oct 30 21:35:13.481214 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:35:13.481191    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac\": container with ID starting with 5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac not found: ID does not exist" containerID="5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac"
Oct 30 21:35:13.481297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.481227    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac"} err="failed to get container status \"5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac\": rpc error: code = NotFound desc = could not find container \"5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac\": container with ID starting with 5398717ebe5e06ff5f028c63c90d8f4e248c066c5f1415de15cdd04103f745ac not found: ID does not exist"
Oct 30 21:35:13.481297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.481250    4549 scope.go:122] "RemoveContainer" containerID="1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"
Oct 30 21:35:13.481531 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:35:13.481513    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b\": container with ID starting with 1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b not found: ID does not exist" containerID="1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"
Oct 30 21:35:13.481577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.481539    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b"} err="failed to get container status \"1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b\": rpc error: code = NotFound desc = could not find container \"1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b\": container with ID starting with 1f078d334d41c52991ed2a82aacad61b3a455b2316d2575629c03d6a2ceba72b not found: ID does not exist"
Oct 30 21:35:13.517851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.517762    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"]
Oct 30 21:35:13.521987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.521958    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt"]
Oct 30 21:35:13.628750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:13.628718    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:15.226990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:15.226950    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:15.348438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:15.348406    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22f11dbc-7817-471f-b839-0b277254e60b" path="/var/lib/kubelet/pods/22f11dbc-7817-471f-b839-0b277254e60b/volumes"
Oct 30 21:35:16.781416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:16.781380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:18.501005 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:18.500972    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:35:18.505021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:18.505000    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:35:18.538711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:18.538681    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:20.158858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:20.155456    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:21.868213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:21.868175    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:23.515654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:23.515458    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:25.188340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:25.188292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:26.575685 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:35:26.575646    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:35:26.952515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:26.952408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:29.042413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:29.035761    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:30.935995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:30.935955    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:32.112046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:32.112005    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:33.028549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:33.028512    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:35.070489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:35.070451    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:35.373808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:35.373706    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:37.204787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:37.204732    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:38.795927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:38.795884    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:38.955701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:38.955660    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:40.595353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:40.595281    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:42.775747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:42.774969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:42.953913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:42.953881    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"]
Oct 30 21:35:43.132056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:43.132002    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:43.388708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:43.388619    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:44.673368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:44.655405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.263928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:46.263193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:47.955054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:47.955012    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" containerName="payload-pre-processing" containerID="cri-o://01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a" gracePeriod=25
Oct 30 21:35:48.075700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:48.075571    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:49.610918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:49.610875    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:51.243126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:51.243074    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:53.254425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:53.254380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:54.885861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:54.880568    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:55.486739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:55.486692    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:56.500933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:56.500893    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:35:56.576740 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:35:56.576698    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:35:56.826084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:56.826056    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:58.320001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:58.319967    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:58.945934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:35:58.945888    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:00.534380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:00.534347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:01.619238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:01.619204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:03.243904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:03.243869    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:04.930131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:04.930093    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:06.501335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:06.501299    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:36:06.562090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:06.562050    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:08.034092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.034059    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:08.370229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.370204    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:36:08.399314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.399285    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume\") pod \"c4483c37-9d44-46e4-9266-8dbb05824159\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") "
Oct 30 21:36:08.399505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.399354    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7\" (UniqueName: \"kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7\") pod \"c4483c37-9d44-46e4-9266-8dbb05824159\" (UID: \"c4483c37-9d44-46e4-9266-8dbb05824159\") "
Oct 30 21:36:08.399750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.399728    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume" pod "c4483c37-9d44-46e4-9266-8dbb05824159" (UID: "c4483c37-9d44-46e4-9266-8dbb05824159"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:08.400466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.400440    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4483c37-9d44-46e4-9266-8dbb05824159-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:08.402037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.402017    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7" pod "c4483c37-9d44-46e4-9266-8dbb05824159" (UID: "c4483c37-9d44-46e4-9266-8dbb05824159"). InnerVolumeSpecName "kube-api-access-2rpr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:08.501962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.501924    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rpr7\" (UniqueName: \"kubernetes.io/projected/c4483c37-9d44-46e4-9266-8dbb05824159-kube-api-access-2rpr7\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:08.610942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:08.610821    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:09.065541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.065494    4549 generic.go:356] "Generic (PLEG): container finished" podID="c4483c37-9d44-46e4-9266-8dbb05824159" containerID="01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a" exitCode=0
Oct 30 21:36:09.066090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.065593    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"
Oct 30 21:36:09.066090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.065614    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" event={"ID":"c4483c37-9d44-46e4-9266-8dbb05824159","Type":"ContainerDied","Data":"01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a"}
Oct 30 21:36:09.066090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.065679    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6" event={"ID":"c4483c37-9d44-46e4-9266-8dbb05824159","Type":"ContainerDied","Data":"03ae543e1229d2ce7e3cf4ef0bf99f8db8c8361ee3db7e44febf9c9c73132aa9"}
Oct 30 21:36:09.066090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.065711    4549 scope.go:122] "RemoveContainer" containerID="01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a"
Oct 30 21:36:09.076972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.076814    4549 scope.go:122] "RemoveContainer" containerID="01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a"
Oct 30 21:36:09.077136 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:09.077117    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a\": container with ID starting with 01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a not found: ID does not exist" containerID="01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a"
Oct 30 21:36:09.077181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.077145    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a"} err="failed to get container status \"01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a\": rpc error: code = NotFound desc = could not find container \"01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a\": container with ID starting with 01bee8d4eb36477f44b93f00bf4cdadcdb8520964d1a56b26f8a0f1f127a004a not found: ID does not exist"
Oct 30 21:36:09.118726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.118694    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"]
Oct 30 21:36:09.122992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.122965    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6"]
Oct 30 21:36:09.349238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.349141    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" path="/var/lib/kubelet/pods/c4483c37-9d44-46e4-9266-8dbb05824159/volumes"
Oct 30 21:36:09.458298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:09.458268    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:11.151201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:11.151163    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:12.738721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:12.738685    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:14.481974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:14.481936    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:16.015035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:16.015000    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:16.432972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:16.432924    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4" podUID="4bf73f9c-8107-47f1-9533-2aa3fa3e7606" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:36:16.599045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:16.599000    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"]
Oct 30 21:36:16.703919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:16.702785    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:17.543934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:17.543892    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:19.367381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:19.367339    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:20.918138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:20.918104    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:22.488098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:22.488066    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:23.940229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:23.940193    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:25.673668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:25.673637    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:26.577122 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:26.576974    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:36:27.113701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:27.113666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:27.265585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:27.265548    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:27.690855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:27.690582    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"]
Oct 30 21:36:27.690855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:27.690664    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"]
Oct 30 21:36:27.798497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:27.798464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:27.950583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:27.950326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:28.726144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:28.726109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:29.540603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:29.540562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:29.778822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:29.778770    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:30.213392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:30.213354    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:31.629167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:31.629122    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" podUID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" containerName="main" containerID="cri-o://11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c" gracePeriod=45
Oct 30 21:36:31.677033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:31.671124    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:31.947439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:31.947413    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:36:32.036327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036295    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z\" (UniqueName: \"kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z\") pod \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") "
Oct 30 21:36:32.036551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036341    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs\" (UniqueName: \"kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs\") pod \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") "
Oct 30 21:36:32.036551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036387    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home\") pod \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") "
Oct 30 21:36:32.036735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036705    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home" pod "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" (UID: "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:32.036812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036719    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm\") pod \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") "
Oct 30 21:36:32.036923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036881    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache\") pod \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") "
Oct 30 21:36:32.036979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.036934    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir\") pod \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\" (UID: \"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46\") "
Oct 30 21:36:32.037267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.037214    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache" pod "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" (UID: "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:32.037403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.037382    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir" pod "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" (UID: "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:32.038115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.038089    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:32.038290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.038264    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:32.038290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.038292    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:32.039137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.039114    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs" pod "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" (UID: "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:36:32.039228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.039180    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z" pod "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" (UID: "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46"). InnerVolumeSpecName "kube-api-access-rhl9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:32.039754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.039730    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm" pod "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" (UID: "04c3b5cf-40eb-4ec7-8bee-050f87d2fc46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:32.044046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.043998    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:32.140006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.139952    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:32.140006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.139999    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhl9z\" (UniqueName: \"kubernetes.io/projected/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-kube-api-access-rhl9z\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:32.140006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.140010    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:32.704314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.704242    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5" podUID="87577dca-ef17-455d-80b2-8dcf5728f362" containerName="payload-processing" containerID="cri-o://09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13" gracePeriod=25
Oct 30 21:36:32.704749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.704252    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerName="payload-pre-processing" containerID="cri-o://ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740" gracePeriod=25
Oct 30 21:36:32.728531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.728498    4549 generic.go:356] "Generic (PLEG): container finished" podID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" containerID="11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c" exitCode=0
Oct 30 21:36:32.728690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.728569    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"
Oct 30 21:36:32.728690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.728575    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" event={"ID":"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46","Type":"ContainerDied","Data":"11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c"}
Oct 30 21:36:32.728690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.728621    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh" event={"ID":"04c3b5cf-40eb-4ec7-8bee-050f87d2fc46","Type":"ContainerDied","Data":"a4e435bdc62d49c22cd256d1aecb2f6723fa89da72ddc400abdb232a1d596144"}
Oct 30 21:36:32.728690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.728641    4549 scope.go:122] "RemoveContainer" containerID="11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c"
Oct 30 21:36:32.818878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.818859    4549 scope.go:122] "RemoveContainer" containerID="11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c"
Oct 30 21:36:32.819221 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:32.819198    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c\": container with ID starting with 11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c not found: ID does not exist" containerID="11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c"
Oct 30 21:36:32.819300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.819233    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c"} err="failed to get container status \"11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c\": rpc error: code = NotFound desc = could not find container \"11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c\": container with ID starting with 11a985f7527b82a682e49dcccf578db4b9d000f1c5636c3ae6a5ec41c570950c not found: ID does not exist"
Oct 30 21:36:32.837259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.837223    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"]
Oct 30 21:36:32.842684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:32.842655    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-200d11/test-model-200d11-kserve-748cb55d75-rmmfh"]
Oct 30 21:36:33.003436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.003405    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:36:33.155698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.155665    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume\") pod \"87577dca-ef17-455d-80b2-8dcf5728f362\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") "
Oct 30 21:36:33.155914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.155812    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s\" (UniqueName: \"kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s\") pod \"87577dca-ef17-455d-80b2-8dcf5728f362\" (UID: \"87577dca-ef17-455d-80b2-8dcf5728f362\") "
Oct 30 21:36:33.156111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.156082    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume" pod "87577dca-ef17-455d-80b2-8dcf5728f362" (UID: "87577dca-ef17-455d-80b2-8dcf5728f362"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:33.156552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.156530    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87577dca-ef17-455d-80b2-8dcf5728f362-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:33.158178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.158155    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s" pod "87577dca-ef17-455d-80b2-8dcf5728f362" (UID: "87577dca-ef17-455d-80b2-8dcf5728f362"). InnerVolumeSpecName "kube-api-access-lbm5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:33.258632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.258541    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbm5s\" (UniqueName: \"kubernetes.io/projected/87577dca-ef17-455d-80b2-8dcf5728f362-kube-api-access-lbm5s\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:33.342139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.342097    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:33.383858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.375103    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" path="/var/lib/kubelet/pods/04c3b5cf-40eb-4ec7-8bee-050f87d2fc46/volumes"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.420411    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"]
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422229    4549 cpu_manager.go:414] "containerMap: removing container" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422255    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422276    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" containerName="payload-pre-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422285    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" containerName="payload-pre-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422295    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422304    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422318    4549 cpu_manager.go:414] "containerMap: removing container" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422326    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422354    4549 cpu_manager.go:414] "containerMap: removing container" podUID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" containerName="main"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422362    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" containerName="main"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422386    4549 cpu_manager.go:414] "containerMap: removing container" podUID="87577dca-ef17-455d-80b2-8dcf5728f362" containerName="payload-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422394    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87577dca-ef17-455d-80b2-8dcf5728f362" containerName="payload-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422413    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" containerName="main"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422419    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" containerName="main"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422445    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerName="payload-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422455    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerName="payload-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422485    4549 cpu_manager.go:414] "containerMap: removing container" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerName="payload-pre-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422495    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerName="payload-pre-processing"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422510    4549 cpu_manager.go:414] "containerMap: removing container" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy"
Oct 30 21:36:33.422670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422521    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422711    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422724    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f2dc276-80c4-48f1-9a81-2e5197378916" containerName="payload-processing"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422737    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="87fda6f5-520b-4855-a3d6-de68f43bf1c8" containerName="payload-pre-processing"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422748    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="123cd94c-22ac-4613-92c7-be45db78256c" containerName="istio-proxy"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422762    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422771    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4483c37-9d44-46e4-9266-8dbb05824159" containerName="payload-pre-processing"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422779    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422788    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="22f11dbc-7817-471f-b839-0b277254e60b" containerName="istio-proxy"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422813    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="04c3b5cf-40eb-4ec7-8bee-050f87d2fc46" containerName="main"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422844    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="87577dca-ef17-455d-80b2-8dcf5728f362" containerName="payload-processing"
Oct 30 21:36:33.424991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.422855    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e745e0e6-7871-4b4d-8f3b-241d3a83b4ea" containerName="main"
Oct 30 21:36:33.448815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.448784    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.466215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.466179    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"]
Oct 30 21:36:33.474074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.474036    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"]
Oct 30 21:36:33.474279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.474247    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" containerID="cri-o://dadd5b6bda2e5104c26408bf21ee18763ac80acda4a5e7348d8425bb0484600f" gracePeriod=30
Oct 30 21:36:33.475897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.474905    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-a4782a-openshift-default-dockercfg-5x6tr\""
Oct 30 21:36:33.563249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563145    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563295    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563347    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563418    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563475    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563564    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563673    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563802    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fzsf\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.563965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.563952    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665232    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665309    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fzsf\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665396    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665416    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665476    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665519    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665587    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665616    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665704    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.665933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.665907    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.666100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.666070    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.666231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.666147    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.666231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.666165    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.666518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.666278    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.668084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.668065    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.668223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.668207    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.686584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.686561    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.686721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.686630    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fzsf\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf\") pod \"e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") " pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.737549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.737513    4549 generic.go:356] "Generic (PLEG): container finished" podID="87577dca-ef17-455d-80b2-8dcf5728f362" containerID="09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13" exitCode=0
Oct 30 21:36:33.738058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.737559    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"
Oct 30 21:36:33.738058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.737581    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5" event={"ID":"87577dca-ef17-455d-80b2-8dcf5728f362","Type":"ContainerDied","Data":"09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13"}
Oct 30 21:36:33.738058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.737639    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5" event={"ID":"87577dca-ef17-455d-80b2-8dcf5728f362","Type":"ContainerDied","Data":"690d4a599b3b1d11a13d2a9399e13e64b6d6eb9d3259c3b8468194d40f7630cd"}
Oct 30 21:36:33.738058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.737668    4549 scope.go:122] "RemoveContainer" containerID="09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13"
Oct 30 21:36:33.749113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.749086    4549 scope.go:122] "RemoveContainer" containerID="09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13"
Oct 30 21:36:33.749424 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:33.749402    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13\": container with ID starting with 09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13 not found: ID does not exist" containerID="09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13"
Oct 30 21:36:33.749505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.749439    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13"} err="failed to get container status \"09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13\": rpc error: code = NotFound desc = could not find container \"09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13\": container with ID starting with 09e9f4958744903e6cab48302aca99d3368deb437905645272dd7033c4258c13 not found: ID does not exist"
Oct 30 21:36:33.767352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.767309    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:33.798061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.798028    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"]
Oct 30 21:36:33.805137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:33.805102    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5"]
Oct 30 21:36:34.107183 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:36:34.107140    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b997d8_5e21_4c24_a638_b193aaf3d240.slice/crio-7f205e13848fe888492706b3b7242c2bb611b38f87d3ce2f31f4159fb5998c86 WatchSource:0}: Error finding container 7f205e13848fe888492706b3b7242c2bb611b38f87d3ce2f31f4159fb5998c86: Status 404 returned error can't find the container with id 7f205e13848fe888492706b3b7242c2bb611b38f87d3ce2f31f4159fb5998c86
Oct 30 21:36:34.108671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.108646    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"]
Oct 30 21:36:34.110444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.110408    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:34.110548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.110476    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:34.110548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.110505    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:34.750157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.750110    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" event={"ID":"b6b997d8-5e21-4c24-a638-b193aaf3d240","Type":"ContainerStarted","Data":"72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c"}
Oct 30 21:36:34.750703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.750219    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" event={"ID":"b6b997d8-5e21-4c24-a638-b193aaf3d240","Type":"ContainerStarted","Data":"7f205e13848fe888492706b3b7242c2bb611b38f87d3ce2f31f4159fb5998c86"}
Oct 30 21:36:34.768229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.768169    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:34.808466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.808401    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podStartSLOduration=1.808382358 podStartE2EDuration="1.808382358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:34.804434547 +0000 UTC m=+2177.578834487" watchObservedRunningTime="2026-10-30 21:36:34.808382358 +0000 UTC m=+2177.582782298"
Oct 30 21:36:34.868923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:34.868877    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:35.067914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:35.064814    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:35.400466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:35.400384    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87577dca-ef17-455d-80b2-8dcf5728f362" path="/var/lib/kubelet/pods/87577dca-ef17-455d-80b2-8dcf5728f362/volumes"
Oct 30 21:36:35.793925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:35.792051    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:36:35.793925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:35.792121    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:36:36.767689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:36.767650    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:36:36.767905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:36.767721    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:36:37.007701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:37.007435    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:37.696887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:37.696847    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:37.769672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:37.769636    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:36:37.770184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:37.770156    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:36:37.842873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:37.842810    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:38.015966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.015794    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"]
Oct 30 21:36:38.016501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.016163    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy" containerID="cri-o://f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12" gracePeriod=30
Oct 30 21:36:38.115784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.115735    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:38.459130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.459095    4549 patch_prober.go:29] interesting pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:36:38.459412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.459388    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:36:38.800499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.799082    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:36:38.800499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.799144    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:36:38.819814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.819772    4549 generic.go:356] "Generic (PLEG): container finished" podID="5867ad60-94d0-4054-8649-135537d67730" containerID="dadd5b6bda2e5104c26408bf21ee18763ac80acda4a5e7348d8425bb0484600f" exitCode=0
Oct 30 21:36:38.819999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.819907    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" event={"ID":"5867ad60-94d0-4054-8649-135537d67730","Type":"ContainerDied","Data":"dadd5b6bda2e5104c26408bf21ee18763ac80acda4a5e7348d8425bb0484600f"}
Oct 30 21:36:38.852460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.852424    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:36:38.954177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.954136    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.954369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.954345    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.954503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.954473    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:38.954778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.954711    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:38.954919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.954849    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:38.955059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.954384    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955169    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955202    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955240    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955268    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955303    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955333    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs\") pod \"5867ad60-94d0-4054-8649-135537d67730\" (UID: \"5867ad60-94d0-4054-8649-135537d67730\") "
Oct 30 21:36:38.955492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.955391    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:38.956371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.956263    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:38.956577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.956556    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:38.956653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.956587    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5867ad60-94d0-4054-8649-135537d67730-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:38.956653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.956621    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:38.956653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.956636    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:38.957950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.957926    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "kube-api-access-vd6gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:38.958074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.958041    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:36:38.958350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.958333    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:38.959076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:38.959045    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy" pod "5867ad60-94d0-4054-8649-135537d67730" (UID: "5867ad60-94d0-4054-8649-135537d67730"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:39.062928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.059245    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:39.062928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.059281    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:39.062928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.059295    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5867ad60-94d0-4054-8649-135537d67730-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:39.062928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.059307    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd6gv\" (UniqueName: \"kubernetes.io/projected/5867ad60-94d0-4054-8649-135537d67730-kube-api-access-vd6gv\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:39.062928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.059319    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5867ad60-94d0-4054-8649-135537d67730-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:39.200108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.200007    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:39.767738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.767652    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:36:39.767917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.767728    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:36:39.828946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.828918    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"
Oct 30 21:36:39.828946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.828933    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64" event={"ID":"5867ad60-94d0-4054-8649-135537d67730","Type":"ContainerDied","Data":"dcac1fb31e72281fd68b87d6510d4a775037e17ae19ced909729011d41bde41f"}
Oct 30 21:36:39.829189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.829027    4549 scope.go:122] "RemoveContainer" containerID="dadd5b6bda2e5104c26408bf21ee18763ac80acda4a5e7348d8425bb0484600f"
Oct 30 21:36:39.897845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.897798    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"]
Oct 30 21:36:39.902021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:39.901991    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64"]
Oct 30 21:36:40.613509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:40.613461    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:36:40.767914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:40.767877    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:36:40.768103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:40.767941    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:36:41.088308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.088272    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:41.346969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.346879    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5867ad60-94d0-4054-8649-135537d67730" path="/var/lib/kubelet/pods/5867ad60-94d0-4054-8649-135537d67730/volumes"
Oct 30 21:36:41.773252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.773171    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:41.773712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.773690    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:41.774631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.774610    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:36:41.933070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.933033    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"]
Oct 30 21:36:41.934293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.934266    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy"
Oct 30 21:36:41.934293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.934292    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2be14cc1-2729-4b2a-a37f-b01dd0b27a84" containerName="istio-proxy"
Oct 30 21:36:41.934504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.934312    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy"
Oct 30 21:36:41.934504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.934321    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy"
Oct 30 21:36:41.934608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.934550    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5867ad60-94d0-4054-8649-135537d67730" containerName="istio-proxy"
Oct 30 21:36:41.939134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.939110    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:41.964326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.964296    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-abf04b-openshift-default-dockercfg-knqgx\""
Oct 30 21:36:41.977938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:41.977904    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"]
Oct 30 21:36:42.032741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.032602    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.039135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.038996    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.039599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.039536    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j57bq\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.039980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.039958    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.040178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.040163    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.040440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.040412    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.040647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.040630    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.040815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.040802    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.041235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.041214    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143044    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143099    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143143    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143332    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143368    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143424    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143478    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j57bq\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143537    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143559    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143578    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143626    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.143978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.143711    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.144264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.144057    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.144333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.144315    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.146439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.146421    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.146575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.146555    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.163820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.163789    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.163997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.163936    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j57bq\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq\") pod \"e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") " pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.260636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.260602    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:42.589548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.589503    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"]
Oct 30 21:36:42.589804 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:36:42.589782    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87603398_cad5_4a8a_bea0_829f0d4acb2f.slice/crio-7da0a7b3aab2d7d77e070ffb7eebbc2facff1fa86b6af81f588c4bf7399e35d7 WatchSource:0}: Error finding container 7da0a7b3aab2d7d77e070ffb7eebbc2facff1fa86b6af81f588c4bf7399e35d7: Status 404 returned error can't find the container with id 7da0a7b3aab2d7d77e070ffb7eebbc2facff1fa86b6af81f588c4bf7399e35d7
Oct 30 21:36:42.593177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.593143    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:42.593267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.593209    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:42.593267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.593241    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:42.763745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.763707    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:42.858594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.858508    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" event={"ID":"87603398-cad5-4a8a-bea0-829f0d4acb2f","Type":"ContainerStarted","Data":"dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac"}
Oct 30 21:36:42.858594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.858562    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" event={"ID":"87603398-cad5-4a8a-bea0-829f0d4acb2f","Type":"ContainerStarted","Data":"7da0a7b3aab2d7d77e070ffb7eebbc2facff1fa86b6af81f588c4bf7399e35d7"}
Oct 30 21:36:42.918151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:42.918068    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podStartSLOduration=1.9180494449999999 podStartE2EDuration="1.918049445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:42.91620493 +0000 UTC m=+2185.690604883" watchObservedRunningTime="2026-10-30 21:36:42.918049445 +0000 UTC m=+2185.692449386"
Oct 30 21:36:43.261308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.261265    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:43.262620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.262591    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:43.262752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.262646    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:43.320359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.320046    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:36:43.367195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367158    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367366    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367407    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367445    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367482    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:43.367624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367538    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367622    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367683    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:43.367741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367711    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367744    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.367941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367923    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs\") pod \"16e9ef29-3164-4072-9560-251a9db10da5\" (UID: \"16e9ef29-3164-4072-9560-251a9db10da5\") "
Oct 30 21:36:43.368007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.367953    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:43.368921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.368768    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.368921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.368795    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.368921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.368807    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e9ef29-3164-4072-9560-251a9db10da5-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.369179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.368984    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:43.369179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.369014    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:43.370610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.370579    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:36:43.371322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.371296    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:43.371425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.371390    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:43.371679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.371659    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv" pod "16e9ef29-3164-4072-9560-251a9db10da5" (UID: "16e9ef29-3164-4072-9560-251a9db10da5"). InnerVolumeSpecName "kube-api-access-xj5rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:43.470912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.470877    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.470912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.470905    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e9ef29-3164-4072-9560-251a9db10da5-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.470912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.470915    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.471152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.470925    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.471152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.470933    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj5rv\" (UniqueName: \"kubernetes.io/projected/16e9ef29-3164-4072-9560-251a9db10da5-kube-api-access-xj5rv\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.471152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.470941    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e9ef29-3164-4072-9560-251a9db10da5-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:43.867666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.867632    4549 generic.go:356] "Generic (PLEG): container finished" podID="16e9ef29-3164-4072-9560-251a9db10da5" containerID="f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12" exitCode=0
Oct 30 21:36:43.868115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.867683    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"
Oct 30 21:36:43.868115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.867736    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" event={"ID":"16e9ef29-3164-4072-9560-251a9db10da5","Type":"ContainerDied","Data":"f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12"}
Oct 30 21:36:43.868115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.867788    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl" event={"ID":"16e9ef29-3164-4072-9560-251a9db10da5","Type":"ContainerDied","Data":"049aa7c4e65b1d6fed00af352fc779931112d5e0de42d034cc680692b6d927f6"}
Oct 30 21:36:43.868115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.867814    4549 scope.go:122] "RemoveContainer" containerID="f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12"
Oct 30 21:36:43.905192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.905172    4549 scope.go:122] "RemoveContainer" containerID="f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12"
Oct 30 21:36:43.905532 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:43.905506    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12\": container with ID starting with f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12 not found: ID does not exist" containerID="f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12"
Oct 30 21:36:43.905630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.905540    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12"} err="failed to get container status \"f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12\": rpc error: code = NotFound desc = could not find container \"f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12\": container with ID starting with f3034b1017b82075d891d04f7b37d7fb1c2d0e94a5b2f0a88a6fa8486f3b3d12 not found: ID does not exist"
Oct 30 21:36:43.951751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.951721    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"]
Oct 30 21:36:43.965252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:43.965222    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl"]
Oct 30 21:36:44.261280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:44.261197    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:44.261280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:44.261258    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:44.394615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:44.394576    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:44.592858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:44.580519    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:45.265270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:45.265236    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:45.265672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:45.265308    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:45.349662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:45.349604    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16e9ef29-3164-4072-9560-251a9db10da5" path="/var/lib/kubelet/pods/16e9ef29-3164-4072-9560-251a9db10da5/volumes"
Oct 30 21:36:46.111872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:46.108941    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:46.262067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:46.262031    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:46.262225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:46.262083    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:47.261319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:47.261281    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:47.261789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:47.261338    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:47.619931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:47.619823    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:48.261143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:48.261104    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:48.261339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:48.261163    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:49.216627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:49.216593    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"]
Oct 30 21:36:49.261512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:49.261477    4549 patch_prober.go:29] interesting pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:36:49.261733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:49.261539    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:36:49.269041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:49.269010    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"]
Oct 30 21:36:49.323991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:49.323935    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:49.478908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:49.477757    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:50.134307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:50.134275    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"]
Oct 30 21:36:50.134512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:50.134473    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" containerID="cri-o://8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48" gracePeriod=30
Oct 30 21:36:50.270141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:50.270107    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:50.270596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:50.270543    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:50.271387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:50.271368    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:36:50.614399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:50.614355    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:36:51.338728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:51.338683    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:51.540418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:51.538419    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:52.436454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:52.436413    4549 patch_prober.go:29] interesting pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:36:52.466968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:52.466919    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:36:53.020962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.020929    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:36:53.060844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.060796    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h\" (UniqueName: \"kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h\") pod \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") "
Oct 30 21:36:53.061042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.061013    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume\") pod \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\" (UID: \"4407ac7e-75a7-4990-9585-8d4f9502e5e0\") "
Oct 30 21:36:53.061450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.061423    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume" pod "4407ac7e-75a7-4990-9585-8d4f9502e5e0" (UID: "4407ac7e-75a7-4990-9585-8d4f9502e5e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:53.061715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.061699    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4407ac7e-75a7-4990-9585-8d4f9502e5e0-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:53.063162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.063136    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h" pod "4407ac7e-75a7-4990-9585-8d4f9502e5e0" (UID: "4407ac7e-75a7-4990-9585-8d4f9502e5e0"). InnerVolumeSpecName "kube-api-access-w4m4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:53.086618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.086571    4549 generic.go:356] "Generic (PLEG): container finished" podID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerID="ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740" exitCode=0
Oct 30 21:36:53.086788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.086699    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" event={"ID":"4407ac7e-75a7-4990-9585-8d4f9502e5e0","Type":"ContainerDied","Data":"ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740"}
Oct 30 21:36:53.086788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.086752    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd" event={"ID":"4407ac7e-75a7-4990-9585-8d4f9502e5e0","Type":"ContainerDied","Data":"51ef1ff7c4bb22ebfe4c63288d526f55788cffdc521a5b75ac933237a4584c27"}
Oct 30 21:36:53.086788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.086782    4549 scope.go:122] "RemoveContainer" containerID="ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740"
Oct 30 21:36:53.087038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.086972    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"
Oct 30 21:36:53.102428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.102402    4549 scope.go:122] "RemoveContainer" containerID="ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740"
Oct 30 21:36:53.102824 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:53.102787    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740\": container with ID starting with ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740 not found: ID does not exist" containerID="ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740"
Oct 30 21:36:53.102920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.102852    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740"} err="failed to get container status \"ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740\": rpc error: code = NotFound desc = could not find container \"ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740\": container with ID starting with ebf91f7c33a1ff96b38b17ca17bdb1fd02558ca3d0934072de8a3eb57cac3740 not found: ID does not exist"
Oct 30 21:36:53.146316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.146272    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"]
Oct 30 21:36:53.153501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.153472    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd"]
Oct 30 21:36:53.163996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.163957    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4m4h\" (UniqueName: \"kubernetes.io/projected/4407ac7e-75a7-4990-9585-8d4f9502e5e0-kube-api-access-w4m4h\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:53.345917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.345885    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" path="/var/lib/kubelet/pods/4407ac7e-75a7-4990-9585-8d4f9502e5e0/volumes"
Oct 30 21:36:53.574534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:53.573971    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:54.217131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:54.217086    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerName="payload-processing" containerID="cri-o://e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be" gracePeriod=25
Oct 30 21:36:54.269320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:54.269253    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" containerName="payload-pre-processing" containerID="cri-o://843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219" gracePeriod=25
Oct 30 21:36:55.106582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.106533    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:55.436915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.436886    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:36:55.498146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498112    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498215    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498251    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498285    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498348    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498383    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498467    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498495    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498565    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:55.498851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498645    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy\") pod \"7ebd63a4-8699-4329-9480-3e022170dee4\" (UID: \"7ebd63a4-8699-4329-9480-3e022170dee4\") "
Oct 30 21:36:55.498851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498672    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:55.498851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498731    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:55.499017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498915    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:55.499017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.498965    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:55.499286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.499263    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ebd63a4-8699-4329-9480-3e022170dee4-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.499286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.499288    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.499497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.499303    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.499497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.499316    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.499497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.499329    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.501153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.501108    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:55.501623 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.501602    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:36:55.501712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.501697    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:36:55.503172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.503148    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj" pod "7ebd63a4-8699-4329-9480-3e022170dee4" (UID: "7ebd63a4-8699-4329-9480-3e022170dee4"). InnerVolumeSpecName "kube-api-access-85kpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:55.600656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.600618    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ebd63a4-8699-4329-9480-3e022170dee4-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.600656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.600646    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.600656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.600655    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ebd63a4-8699-4329-9480-3e022170dee4-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:55.600656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:55.600664    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85kpj\" (UniqueName: \"kubernetes.io/projected/7ebd63a4-8699-4329-9480-3e022170dee4-kube-api-access-85kpj\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:36:56.113300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.113263    4549 generic.go:356] "Generic (PLEG): container finished" podID="7ebd63a4-8699-4329-9480-3e022170dee4" containerID="8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48" exitCode=0
Oct 30 21:36:56.113732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.113330    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"
Oct 30 21:36:56.113732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.113339    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" event={"ID":"7ebd63a4-8699-4329-9480-3e022170dee4","Type":"ContainerDied","Data":"8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48"}
Oct 30 21:36:56.113732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.113399    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt" event={"ID":"7ebd63a4-8699-4329-9480-3e022170dee4","Type":"ContainerDied","Data":"e3a1e0d17733f3f8d26ee59c9187dba850878484fd8a301aabf6349011c847bc"}
Oct 30 21:36:56.113732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.113426    4549 scope.go:122] "RemoveContainer" containerID="8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48"
Oct 30 21:36:56.125380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.125359    4549 scope.go:122] "RemoveContainer" containerID="8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48"
Oct 30 21:36:56.125681 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:56.125662    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48\": container with ID starting with 8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48 not found: ID does not exist" containerID="8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48"
Oct 30 21:36:56.125720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.125692    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48"} err="failed to get container status \"8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48\": rpc error: code = NotFound desc = could not find container \"8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48\": container with ID starting with 8d700746fd6d1003f84d3ebf547674d0892072b8c7cb33fe65fae32809a05b48 not found: ID does not exist"
Oct 30 21:36:56.178368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.178333    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"]
Oct 30 21:36:56.190309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.190275    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt"]
Oct 30 21:36:56.576053 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:36:56.576014    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:36:56.687800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:56.687762    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:57.022673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:57.022579    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:36:57.022673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:57.022599    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:36:57.349184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:57.349144    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" path="/var/lib/kubelet/pods/7ebd63a4-8699-4329-9480-3e022170dee4/volumes"
Oct 30 21:36:58.063714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:58.063677    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:58.718590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:58.718394    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:59.062467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:59.062274    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:59.203437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:59.203401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:59.401051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:36:59.401019    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:00.805768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:00.805734    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:02.251354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:02.251322    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:03.488469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:03.488436    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:03.985435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:03.985352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:04.236889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:04.236733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:04.392323 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:04.392266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:04.779450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:04.779383    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:06.071043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:06.070994    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:06.379871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:06.379736    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"]
Oct 30 21:37:06.461427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:06.461389    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:06.639676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:06.639588    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:06.891917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:06.891152    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:07.088100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:07.088059    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:37:07.089122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:07.089086    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:37:07.862859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:07.850420    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:08.743008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:08.742969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:09.535558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:09.535520    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:10.930701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:10.930663    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:11.068875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.068823    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:11.207461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.207365    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"]
Oct 30 21:37:11.207670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.207641    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy" containerID="cri-o://01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee" gracePeriod=30
Oct 30 21:37:11.217074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.217046    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"]
Oct 30 21:37:11.217277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.217249    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy" containerID="cri-o://c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580" gracePeriod=30
Oct 30 21:37:11.220169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.220128    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:11.448926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:11.448890    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:12.914050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:12.914007    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:14.478665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.475659    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:14.624927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.624889    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:37:14.676522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.676497    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:37:14.781573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.781481    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw\" (UniqueName: \"kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw\") pod \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") "
Oct 30 21:37:14.781742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.781695    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume\") pod \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\" (UID: \"53b1a8bf-adc9-44a6-bcf0-510c985a735a\") "
Oct 30 21:37:14.781798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.781766    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr\" (UniqueName: \"kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr\") pod \"537612db-c727-42c3-b8a7-febe5d6e1812\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") "
Oct 30 21:37:14.781896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.781873    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume\") pod \"537612db-c727-42c3-b8a7-febe5d6e1812\" (UID: \"537612db-c727-42c3-b8a7-febe5d6e1812\") "
Oct 30 21:37:14.782264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.782232    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume" pod "53b1a8bf-adc9-44a6-bcf0-510c985a735a" (UID: "53b1a8bf-adc9-44a6-bcf0-510c985a735a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:14.782264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.782262    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume" pod "537612db-c727-42c3-b8a7-febe5d6e1812" (UID: "537612db-c727-42c3-b8a7-febe5d6e1812"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:14.782765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.782745    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53b1a8bf-adc9-44a6-bcf0-510c985a735a-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:14.782765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.782765    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/537612db-c727-42c3-b8a7-febe5d6e1812-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:14.784678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.784653    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw" pod "53b1a8bf-adc9-44a6-bcf0-510c985a735a" (UID: "53b1a8bf-adc9-44a6-bcf0-510c985a735a"). InnerVolumeSpecName "kube-api-access-5lzcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:14.784774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.784697    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr" pod "537612db-c727-42c3-b8a7-febe5d6e1812" (UID: "537612db-c727-42c3-b8a7-febe5d6e1812"). InnerVolumeSpecName "kube-api-access-ppgdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:14.884499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.884457    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lzcw\" (UniqueName: \"kubernetes.io/projected/53b1a8bf-adc9-44a6-bcf0-510c985a735a-kube-api-access-5lzcw\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:14.884499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:14.884493    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppgdr\" (UniqueName: \"kubernetes.io/projected/537612db-c727-42c3-b8a7-febe5d6e1812-kube-api-access-ppgdr\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:15.271015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.270977    4549 generic.go:356] "Generic (PLEG): container finished" podID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerID="e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be" exitCode=0
Oct 30 21:37:15.271189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.271010    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" event={"ID":"53b1a8bf-adc9-44a6-bcf0-510c985a735a","Type":"ContainerDied","Data":"e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be"}
Oct 30 21:37:15.271189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.271065    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg" event={"ID":"53b1a8bf-adc9-44a6-bcf0-510c985a735a","Type":"ContainerDied","Data":"396d8d7a7762451e127410e35bb76dd0cdf90b3a96d4c86c4fed5d39d0bf72cc"}
Oct 30 21:37:15.271189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.271086    4549 scope.go:122] "RemoveContainer" containerID="e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be"
Oct 30 21:37:15.271189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.271020    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"
Oct 30 21:37:15.272824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.272794    4549 generic.go:356] "Generic (PLEG): container finished" podID="537612db-c727-42c3-b8a7-febe5d6e1812" containerID="843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219" exitCode=0
Oct 30 21:37:15.272986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.272869    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"
Oct 30 21:37:15.273070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.272907    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" event={"ID":"537612db-c727-42c3-b8a7-febe5d6e1812","Type":"ContainerDied","Data":"843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219"}
Oct 30 21:37:15.273114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.273101    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx" event={"ID":"537612db-c727-42c3-b8a7-febe5d6e1812","Type":"ContainerDied","Data":"732980f7d12f7040524b747a60ff87eb8001b3079df9d118679ca674a2c0ad49"}
Oct 30 21:37:15.287022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.286998    4549 scope.go:122] "RemoveContainer" containerID="e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be"
Oct 30 21:37:15.287404 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:15.287384    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be\": container with ID starting with e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be not found: ID does not exist" containerID="e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be"
Oct 30 21:37:15.287503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.287416    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be"} err="failed to get container status \"e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be\": rpc error: code = NotFound desc = could not find container \"e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be\": container with ID starting with e296681382bcabcf559c59604c7a8fae3e1d308a3c257121895ba7ed860c73be not found: ID does not exist"
Oct 30 21:37:15.287503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.287440    4549 scope.go:122] "RemoveContainer" containerID="843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219"
Oct 30 21:37:15.298360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.298343    4549 scope.go:122] "RemoveContainer" containerID="843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219"
Oct 30 21:37:15.298676 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:15.298655    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219\": container with ID starting with 843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219 not found: ID does not exist" containerID="843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219"
Oct 30 21:37:15.298737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.298684    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219"} err="failed to get container status \"843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219\": rpc error: code = NotFound desc = could not find container \"843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219\": container with ID starting with 843a56d4391329605737ddef67fcee2a772a48eff66a33790c89dd4df59f4219 not found: ID does not exist"
Oct 30 21:37:15.334986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.334941    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"]
Oct 30 21:37:15.346952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.346925    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg"]
Oct 30 21:37:15.380037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.380000    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"]
Oct 30 21:37:15.388345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:15.388315    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx"]
Oct 30 21:37:16.027437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.027403    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:16.547061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.547035    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:37:16.600770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.600738    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:37:16.726148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726114    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.726361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726162    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.726361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726188    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.726361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726212    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.726550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726417    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.726613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726545    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.726668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726600    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.726737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726677    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.726737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726697    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.726737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726707    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:16.727009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726705    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.727009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726818    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.727009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726944    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.727009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.726964    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.727219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727038    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.727219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727128    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.727219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727193    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.727394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727271    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.727394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727317    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:16.727510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727484    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.727619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727600    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo\") pod \"6d79a5cf-d13a-4723-9ea2-517205e31827\" (UID: \"6d79a5cf-d13a-4723-9ea2-517205e31827\") "
Oct 30 21:37:16.727694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727650    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.727694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727681    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket\") pod \"59e347c6-5c61-4579-93c3-f6a576f6cb42\" (UID: \"59e347c6-5c61-4579-93c3-f6a576f6cb42\") "
Oct 30 21:37:16.727802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727733    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.727907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727843    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.727907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.727859    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.728187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728060    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728306    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728663    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59e347c6-5c61-4579-93c3-f6a576f6cb42-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728684    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728700    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728713    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728726    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728756    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728773    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d79a5cf-d13a-4723-9ea2-517205e31827-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728786    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728799    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.728861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.728814    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.730363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730174    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.730363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730305    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "kube-api-access-tfv8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:16.730363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730332    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:16.730363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730339    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "kube-api-access-kmgs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:16.730607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730456    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:16.730917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730899    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token" pod "59e347c6-5c61-4579-93c3-f6a576f6cb42" (UID: "59e347c6-5c61-4579-93c3-f6a576f6cb42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:16.731001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.730929    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:16.731595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.731576    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo" pod "6d79a5cf-d13a-4723-9ea2-517205e31827" (UID: "6d79a5cf-d13a-4723-9ea2-517205e31827"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:16.830733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830686    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.830733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830730    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.830733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830740    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.831079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830750    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmgs2\" (UniqueName: \"kubernetes.io/projected/6d79a5cf-d13a-4723-9ea2-517205e31827-kube-api-access-kmgs2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.831079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830760    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.831079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830768    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfv8t\" (UniqueName: \"kubernetes.io/projected/59e347c6-5c61-4579-93c3-f6a576f6cb42-kube-api-access-tfv8t\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.831079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830777    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d79a5cf-d13a-4723-9ea2-517205e31827-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:16.831079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:16.830785    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59e347c6-5c61-4579-93c3-f6a576f6cb42-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:17.278712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.278662    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:17.292501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.292463    4549 generic.go:356] "Generic (PLEG): container finished" podID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerID="01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee" exitCode=0
Oct 30 21:37:17.292698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.292505    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"
Oct 30 21:37:17.292698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.292505    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" event={"ID":"59e347c6-5c61-4579-93c3-f6a576f6cb42","Type":"ContainerDied","Data":"01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee"}
Oct 30 21:37:17.292698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.292597    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx" event={"ID":"59e347c6-5c61-4579-93c3-f6a576f6cb42","Type":"ContainerDied","Data":"b0086d75fe7d84b1549971f7e40074ee6e60d99820aad0a64de6f5846aa025da"}
Oct 30 21:37:17.292698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.292628    4549 scope.go:122] "RemoveContainer" containerID="01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee"
Oct 30 21:37:17.294811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.294782    4549 generic.go:356] "Generic (PLEG): container finished" podID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerID="c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580" exitCode=0
Oct 30 21:37:17.294967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.294813    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" event={"ID":"6d79a5cf-d13a-4723-9ea2-517205e31827","Type":"ContainerDied","Data":"c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580"}
Oct 30 21:37:17.294967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.294898    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"
Oct 30 21:37:17.295099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.294972    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l" event={"ID":"6d79a5cf-d13a-4723-9ea2-517205e31827","Type":"ContainerDied","Data":"36e482ecfdeeddde6f0fbeec9a4e0aca2fb1760a824b4d82cddbdc3e28f29c2b"}
Oct 30 21:37:17.307873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.307814    4549 scope.go:122] "RemoveContainer" containerID="01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee"
Oct 30 21:37:17.309057 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:17.309036    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee\": container with ID starting with 01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee not found: ID does not exist" containerID="01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee"
Oct 30 21:37:17.309151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.309072    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee"} err="failed to get container status \"01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee\": rpc error: code = NotFound desc = could not find container \"01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee\": container with ID starting with 01b17414d303e86e1c078d9c530b5dd57f79623bed0d8fb55ca6cce2568eafee not found: ID does not exist"
Oct 30 21:37:17.309151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.309096    4549 scope.go:122] "RemoveContainer" containerID="c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580"
Oct 30 21:37:17.320362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.320343    4549 scope.go:122] "RemoveContainer" containerID="e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"
Oct 30 21:37:17.332869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.332824    4549 scope.go:122] "RemoveContainer" containerID="c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580"
Oct 30 21:37:17.333223 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:17.333199    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580\": container with ID starting with c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580 not found: ID does not exist" containerID="c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580"
Oct 30 21:37:17.333286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.333233    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580"} err="failed to get container status \"c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580\": rpc error: code = NotFound desc = could not find container \"c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580\": container with ID starting with c21e0659803bcc395cb353fac788d30a9c5a5bf710f67eb60ea82c9e36167580 not found: ID does not exist"
Oct 30 21:37:17.333286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.333255    4549 scope.go:122] "RemoveContainer" containerID="e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"
Oct 30 21:37:17.333543 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:17.333526    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a\": container with ID starting with e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a not found: ID does not exist" containerID="e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"
Oct 30 21:37:17.333602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.333546    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a"} err="failed to get container status \"e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a\": rpc error: code = NotFound desc = could not find container \"e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a\": container with ID starting with e325fa00ea57cf01bac5a937012c9a3244eb6a70f85ec628b50d178ad187905a not found: ID does not exist"
Oct 30 21:37:17.346788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.346752    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" path="/var/lib/kubelet/pods/537612db-c727-42c3-b8a7-febe5d6e1812/volumes"
Oct 30 21:37:17.347603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.347576    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" path="/var/lib/kubelet/pods/53b1a8bf-adc9-44a6-bcf0-510c985a735a/volumes"
Oct 30 21:37:17.352251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.352227    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"]
Oct 30 21:37:17.363620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.363593    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx"]
Oct 30 21:37:17.642857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.626299    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"]
Oct 30 21:37:17.667006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.666970    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l"]
Oct 30 21:37:17.951569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:17.951464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:18.088057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:18.088023    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:18.216202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:18.216105    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:18.693001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:18.692947    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:19.347419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:19.347384    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" path="/var/lib/kubelet/pods/59e347c6-5c61-4579-93c3-f6a576f6cb42/volumes"
Oct 30 21:37:19.347995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:19.347979    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" path="/var/lib/kubelet/pods/6d79a5cf-d13a-4723-9ea2-517205e31827/volumes"
Oct 30 21:37:19.783121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:19.783089    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:20.356369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:20.356329    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"]
Oct 30 21:37:20.356613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:20.356577    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" containerID="cri-o://a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05" gracePeriod=30
Oct 30 21:37:21.405174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.405113    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerName="main" containerID="cri-o://113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f" gracePeriod=45
Oct 30 21:37:21.535115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.535064    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:21.625513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.625288    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerName="main" probeResult="failure" output="Get \"https://10.133.0.170:8000/ready\": dial tcp 10.133.0.170:8000: connect: connection refused"
Oct 30 21:37:21.764694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.764664    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:37:21.902182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902140    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q\" (UniqueName: \"kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q\") pod \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") "
Oct 30 21:37:21.902366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902211    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm\") pod \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") "
Oct 30 21:37:21.902366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902246    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home\") pod \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") "
Oct 30 21:37:21.902366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902313    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir\") pod \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") "
Oct 30 21:37:21.902366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902353    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache\") pod \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") "
Oct 30 21:37:21.902593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902393    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs\" (UniqueName: \"kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs\") pod \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\" (UID: \"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235\") "
Oct 30 21:37:21.902647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902596    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home" pod "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" (UID: "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:21.902647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902613    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache" pod "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" (UID: "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:21.902647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.902626    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir" pod "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" (UID: "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:21.903095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.903074    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:21.903182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.903098    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:21.903182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.903111    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:21.904807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.904776    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm" pod "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" (UID: "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:21.904959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.904887    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs" pod "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" (UID: "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:37:21.904959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:21.904935    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q" pod "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" (UID: "a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235"). InnerVolumeSpecName "kube-api-access-4lw5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:22.004981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.004868    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:22.004981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.004918    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lw5q\" (UniqueName: \"kubernetes.io/projected/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-kube-api-access-4lw5q\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:22.004981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.004950    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:22.310622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.310525    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:22.340369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.340336    4549 generic.go:356] "Generic (PLEG): container finished" podID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerID="113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f" exitCode=0
Oct 30 21:37:22.340369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.340371    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"
Oct 30 21:37:22.340638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.340411    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" event={"ID":"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235","Type":"ContainerDied","Data":"113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f"}
Oct 30 21:37:22.340638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.340445    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj" event={"ID":"a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235","Type":"ContainerDied","Data":"a341b84092201086ebb6153a5e72c29a2b98d100b198e06187a7a886f7cbd9c3"}
Oct 30 21:37:22.340638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.340465    4549 scope.go:122] "RemoveContainer" containerID="113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f"
Oct 30 21:37:22.354394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.354365    4549 scope.go:122] "RemoveContainer" containerID="113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f"
Oct 30 21:37:22.385857 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:22.379000    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f\": container with ID starting with 113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f not found: ID does not exist" containerID="113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f"
Oct 30 21:37:22.385857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.379074    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f"} err="failed to get container status \"113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f\": rpc error: code = NotFound desc = could not find container \"113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f\": container with ID starting with 113e5ed295d0f6a8d7b76888ac2b24eca23b3d58ed1ef2c55443c209c918830f not found: ID does not exist"
Oct 30 21:37:22.430408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.430375    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"]
Oct 30 21:37:22.442062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.442030    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b2628a/test-model-b2628a-kserve-76c49f4f-g2xrj"]
Oct 30 21:37:22.443006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:22.442980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:23.146145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:23.146101    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:23.346795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:23.346752    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" path="/var/lib/kubelet/pods/a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235/volumes"
Oct 30 21:37:23.743466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:23.743424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:24.823032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:24.822998    4549 patch_prober.go:29] interesting pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:24.823470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:24.823066    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:25.138402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.138304    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:25.659739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.659666    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:37:25.762539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.762501    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:25.792781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.792741    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.792808    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.792904    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.792937    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793009    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793022    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:25.793180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793093    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793155    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793200    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793258    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:25.793399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793293    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:25.793399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793322    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs\") pod \"8d490321-2533-40fc-a4e8-a90521b435b8\" (UID: \"8d490321-2533-40fc-a4e8-a90521b435b8\") "
Oct 30 21:37:25.793593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793401    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:25.793659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.793646    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:25.794300 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.794269    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d490321-2533-40fc-a4e8-a90521b435b8-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.794452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.794305    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.794452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.794322    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.794452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.794338    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.794452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.794352    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.795466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.795437    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:25.795577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.795527    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "kube-api-access-46z2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:25.795577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.795525    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:25.795679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.795663    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token" pod "8d490321-2533-40fc-a4e8-a90521b435b8" (UID: "8d490321-2533-40fc-a4e8-a90521b435b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:25.899186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.899152    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d490321-2533-40fc-a4e8-a90521b435b8-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.899186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.899184    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46z2l\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-kube-api-access-46z2l\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.899186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.899195    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d490321-2533-40fc-a4e8-a90521b435b8-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.899723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.899207    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d490321-2533-40fc-a4e8-a90521b435b8-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:25.903135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:25.903103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:26.020814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.020715    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"]
Oct 30 21:37:26.021050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.021020    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" containerID="cri-o://ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455" gracePeriod=30
Oct 30 21:37:26.082710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.082657    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:26.372745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.372708    4549 generic.go:356] "Generic (PLEG): container finished" podID="8d490321-2533-40fc-a4e8-a90521b435b8" containerID="a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05" exitCode=0
Oct 30 21:37:26.372967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.372764    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"
Oct 30 21:37:26.372967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.372791    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" event={"ID":"8d490321-2533-40fc-a4e8-a90521b435b8","Type":"ContainerDied","Data":"a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05"}
Oct 30 21:37:26.372967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.372883    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v" event={"ID":"8d490321-2533-40fc-a4e8-a90521b435b8","Type":"ContainerDied","Data":"9b7eca8e6b4150a6d9f1104d1e87db511e2978ab949625bd14924711679a38d4"}
Oct 30 21:37:26.372967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.372908    4549 scope.go:122] "RemoveContainer" containerID="a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05"
Oct 30 21:37:26.384789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.384768    4549 scope.go:122] "RemoveContainer" containerID="a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05"
Oct 30 21:37:26.385134 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:26.385112    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05\": container with ID starting with a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05 not found: ID does not exist" containerID="a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05"
Oct 30 21:37:26.385228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.385142    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05"} err="failed to get container status \"a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05\": rpc error: code = NotFound desc = could not find container \"a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05\": container with ID starting with a5d792e6f4b245be826ea7d6a09e6cdb6e186e2eca45129467b64b248c353c05 not found: ID does not exist"
Oct 30 21:37:26.433132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.433102    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"]
Oct 30 21:37:26.448753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.448721    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v"]
Oct 30 21:37:26.577018 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:26.576972    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:37:26.825359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:26.825326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:27.357957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:27.357922    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" path="/var/lib/kubelet/pods/8d490321-2533-40fc-a4e8-a90521b435b8/volumes"
Oct 30 21:37:28.330060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:28.330028    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:29.850346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:29.850305    4549 patch_prober.go:29] interesting pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:29.850774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:29.850363    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:29.923141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:29.923107    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:31.310040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.309963    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:37:31.344196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.344159    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:31.415133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415098    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415276    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415323    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415374    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415428    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415457    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415495    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:31.415677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415658    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415700    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415730    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs\") pod \"a5540263-fc2a-4fc6-b9c4-7045118690ee\" (UID: \"a5540263-fc2a-4fc6-b9c4-7045118690ee\") "
Oct 30 21:37:31.415765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415748    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:31.415985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.415884    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:31.416079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416054    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:31.416192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416170    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:31.416458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416442    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5540263-fc2a-4fc6-b9c4-7045118690ee-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.416552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416466    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.416552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416481    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.416552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416494    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.416552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.416507    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.418256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.418185    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:31.418927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.418903    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "kube-api-access-8mxv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:31.419999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.419974    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:31.423070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.423048    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo" pod "a5540263-fc2a-4fc6-b9c4-7045118690ee" (UID: "a5540263-fc2a-4fc6-b9c4-7045118690ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:31.429048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.429017    4549 generic.go:356] "Generic (PLEG): container finished" podID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerID="ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455" exitCode=0
Oct 30 21:37:31.429158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.429139    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"
Oct 30 21:37:31.429239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.429156    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" event={"ID":"a5540263-fc2a-4fc6-b9c4-7045118690ee","Type":"ContainerDied","Data":"ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455"}
Oct 30 21:37:31.429239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.429209    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm" event={"ID":"a5540263-fc2a-4fc6-b9c4-7045118690ee","Type":"ContainerDied","Data":"9ed066e6648b345cc8a18273dc3469d06a60a65d843f52ff73ea728fce55e706"}
Oct 30 21:37:31.429347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.429239    4549 scope.go:122] "RemoveContainer" containerID="ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455"
Oct 30 21:37:31.461874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.461810    4549 scope.go:122] "RemoveContainer" containerID="90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"
Oct 30 21:37:31.473524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.473504    4549 scope.go:122] "RemoveContainer" containerID="ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455"
Oct 30 21:37:31.473786 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:31.473767    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455\": container with ID starting with ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455 not found: ID does not exist" containerID="ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455"
Oct 30 21:37:31.473921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.473800    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455"} err="failed to get container status \"ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455\": rpc error: code = NotFound desc = could not find container \"ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455\": container with ID starting with ec5f62aa31af63a86766734ff6d35874d21ce304a3b1407a7e8dbd4bb890b455 not found: ID does not exist"
Oct 30 21:37:31.473921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.473823    4549 scope.go:122] "RemoveContainer" containerID="90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"
Oct 30 21:37:31.474127 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:31.474102    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e\": container with ID starting with 90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e not found: ID does not exist" containerID="90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"
Oct 30 21:37:31.474174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.474138    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e"} err="failed to get container status \"90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e\": rpc error: code = NotFound desc = could not find container \"90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e\": container with ID starting with 90de3e8d9f89ac0f427d2fc53d85d18fbdd38c44f12a867b681bf65427db2a3e not found: ID does not exist"
Oct 30 21:37:31.494766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.494740    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"]
Oct 30 21:37:31.500031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.500010    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm"]
Oct 30 21:37:31.518254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.518229    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.518254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.518253    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mxv2\" (UniqueName: \"kubernetes.io/projected/a5540263-fc2a-4fc6-b9c4-7045118690ee-kube-api-access-8mxv2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.518416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.518262    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:31.518416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:31.518272    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5540263-fc2a-4fc6-b9c4-7045118690ee-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:32.292170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:32.292122    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:32.470690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:32.470650    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:32.864967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:32.864932    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.346177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:33.346134    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" path="/var/lib/kubelet/pods/a5540263-fc2a-4fc6-b9c4-7045118690ee/volumes"
Oct 30 21:37:34.297749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:34.297718    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.464629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:34.464596    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.612251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:34.612217    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.123085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:35.123052    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.638666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:35.638623    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.008316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:36.008230    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.264962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:36.264873    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.603021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:36.602988    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.145639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:37.145607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:38.882041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:38.881996    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:40.586958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:40.586929    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.185817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:42.185774    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.358687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:42.358648    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.632264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:42.632234    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"]
Oct 30 21:37:42.632531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:42.632499    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" containerID="cri-o://420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb" gracePeriod=30
Oct 30 21:37:42.688002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:42.687966    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:43.640448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:43.640415    4549 patch_prober.go:29] interesting pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:37:43.640903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:43.640469    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:37:44.085157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:44.085120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:45.762053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:45.762019    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.346789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:47.346750    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"]
Oct 30 21:37:47.367602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:47.367561    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"]
Oct 30 21:37:47.620790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:47.620618    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.831734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:47.831567    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:48.424265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.424238    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:37:48.481545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.481504    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.481752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.481593    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.481841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.481762    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.481914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.481890    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.482032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.482002    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:48.482103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.482022    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.482154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.482121    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.482208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.482172    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.482287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.482269    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.482343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.482322    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket\") pod \"e6ccf846-20db-4a8d-874e-428b2a90d59d\" (UID: \"e6ccf846-20db-4a8d-874e-428b2a90d59d\") "
Oct 30 21:37:48.483118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.483093    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6ccf846-20db-4a8d-874e-428b2a90d59d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.483949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.483906    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:48.484190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.484166    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:48.484302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.484282    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:48.484534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.484496    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "kube-api-access-5p2wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:48.484808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.484778    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:48.486747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.486712    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:48.488545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.487092    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:48.488545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.487289    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo" pod "e6ccf846-20db-4a8d-874e-428b2a90d59d" (UID: "e6ccf846-20db-4a8d-874e-428b2a90d59d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:48.584124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584082    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584116    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584126    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584138    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p2wx\" (UniqueName: \"kubernetes.io/projected/e6ccf846-20db-4a8d-874e-428b2a90d59d-kube-api-access-5p2wx\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584150    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584158    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584166    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.584393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.584174    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6ccf846-20db-4a8d-874e-428b2a90d59d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:37:48.638933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.638806    4549 generic.go:356] "Generic (PLEG): container finished" podID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerID="420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb" exitCode=0
Oct 30 21:37:48.638933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.638903    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" event={"ID":"e6ccf846-20db-4a8d-874e-428b2a90d59d","Type":"ContainerDied","Data":"420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb"}
Oct 30 21:37:48.639162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.638935    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"
Oct 30 21:37:48.639162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.638989    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d" event={"ID":"e6ccf846-20db-4a8d-874e-428b2a90d59d","Type":"ContainerDied","Data":"3c0ec9a15b0970fc8b40937e91638a80aa4c8d698be7cd6d7881f86b3269a887"}
Oct 30 21:37:48.639162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.639023    4549 scope.go:122] "RemoveContainer" containerID="420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb"
Oct 30 21:37:48.651580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.651554    4549 scope.go:122] "RemoveContainer" containerID="108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"
Oct 30 21:37:48.662592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.662570    4549 scope.go:122] "RemoveContainer" containerID="420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb"
Oct 30 21:37:48.662955 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:48.662930    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb\": container with ID starting with 420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb not found: ID does not exist" containerID="420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb"
Oct 30 21:37:48.663063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.662969    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb"} err="failed to get container status \"420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb\": rpc error: code = NotFound desc = could not find container \"420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb\": container with ID starting with 420e41aa272d1d03545ca68874be095fc5a823da4f51800ce3b722d25846cabb not found: ID does not exist"
Oct 30 21:37:48.663063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.662996    4549 scope.go:122] "RemoveContainer" containerID="108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"
Oct 30 21:37:48.663309 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:48.663287    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383\": container with ID starting with 108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383 not found: ID does not exist" containerID="108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"
Oct 30 21:37:48.663352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.663320    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383"} err="failed to get container status \"108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383\": rpc error: code = NotFound desc = could not find container \"108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383\": container with ID starting with 108a86b770267a80e61ccace30bda59b1c2dba2af1eec2f3aced22fd51c83383 not found: ID does not exist"
Oct 30 21:37:48.693573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.693534    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"]
Oct 30 21:37:48.707067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:48.707030    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d"]
Oct 30 21:37:49.083812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:49.083772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.358711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:49.357030    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" path="/var/lib/kubelet/pods/e6ccf846-20db-4a8d-874e-428b2a90d59d/volumes"
Oct 30 21:37:49.672024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:49.671915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.808229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:49.808191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.944202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:49.944116    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:50.283414 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:50.283331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.319211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:51.319168    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.864483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:51.864430    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.027299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:52.027266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.168868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:52.168756    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.347220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:52.347180    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerName="payload-processing" containerID="cri-o://fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc" gracePeriod=25
Oct 30 21:37:52.388911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:52.388865    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerName="payload-pre-processing" containerID="cri-o://3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6" gracePeriod=25
Oct 30 21:37:52.544445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:52.544350    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.763176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:53.763125    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.893855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:53.888577    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:55.235735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:55.235689    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:56.140926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.140880    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:37:56.141143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.140890    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:37:56.380811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.380775    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:56.523688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.523584    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:56.577937 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:37:56.577900    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:37:56.654924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.654889    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:56.805137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.805053    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:56.954052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:56.954017    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:57.179110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:57.179075    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:57.937564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:57.937535    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:58.413004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:58.412972    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:58.584240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:37:58.584192    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:00.022180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:00.022140    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:00.175732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:00.175698    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:00.333619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:00.333580    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"]
Oct 30 21:38:00.333934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:00.333906    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" containerID="cri-o://61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772" gracePeriod=30
Oct 30 21:38:00.369982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:00.369951    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:01.757207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:01.757173    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:02.486204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:02.486155    4549 patch_prober.go:29] interesting pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:38:02.486428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:02.486239    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:38:02.499223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:02.499186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:02.663331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:02.663294    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:03.516701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:03.516665    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.096146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:04.096106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.247896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:04.247862    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:05.131958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.131902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:05.621913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.621885    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:38:05.690900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.690866    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.690978    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691007    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691088    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691119    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691143    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691190    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691272    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691360    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data\") pod \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\" (UID: \"c4b73a33-2d90-4e3d-9680-022ad03c5a2e\") "
Oct 30 21:38:05.691528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691382    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.691528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691416    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:05.691692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691650    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.691965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.691941    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.692140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.692120    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.692214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.692149    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.692214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.692164    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.692214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.692180    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.692401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.692383    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.694939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.694903    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.695231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.695207    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:05.695401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.695335    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:05.695496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.695479    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452" pod "c4b73a33-2d90-4e3d-9680-022ad03c5a2e" (UID: "c4b73a33-2d90-4e3d-9680-022ad03c5a2e"). InnerVolumeSpecName "kube-api-access-rr452". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:05.793152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.793116    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr452\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-kube-api-access-rr452\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.793152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.793149    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.793152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.793159    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.793484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.793168    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.793484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.793178    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b73a33-2d90-4e3d-9680-022ad03c5a2e-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:05.847993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.847958    4549 generic.go:356] "Generic (PLEG): container finished" podID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerID="61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772" exitCode=0
Oct 30 21:38:05.848161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.848032    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"
Oct 30 21:38:05.848161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.848040    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" event={"ID":"c4b73a33-2d90-4e3d-9680-022ad03c5a2e","Type":"ContainerDied","Data":"61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772"}
Oct 30 21:38:05.848161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.848092    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl" event={"ID":"c4b73a33-2d90-4e3d-9680-022ad03c5a2e","Type":"ContainerDied","Data":"25700af00e6e7c66e1912b64f3e54c65c4acca5cc78622960b974a542314c530"}
Oct 30 21:38:05.848161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.848114    4549 scope.go:122] "RemoveContainer" containerID="61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772"
Oct 30 21:38:05.859651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.859624    4549 scope.go:122] "RemoveContainer" containerID="31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"
Oct 30 21:38:05.870069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.870051    4549 scope.go:122] "RemoveContainer" containerID="61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772"
Oct 30 21:38:05.870334 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:05.870314    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772\": container with ID starting with 61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772 not found: ID does not exist" containerID="61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772"
Oct 30 21:38:05.870386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.870344    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772"} err="failed to get container status \"61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772\": rpc error: code = NotFound desc = could not find container \"61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772\": container with ID starting with 61106238f8d0f4d5b512f5090f612c2a0dd5848fe0bad8b8a53aab7514bef772 not found: ID does not exist"
Oct 30 21:38:05.870386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.870362    4549 scope.go:122] "RemoveContainer" containerID="31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"
Oct 30 21:38:05.870606 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:05.870589    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0\": container with ID starting with 31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0 not found: ID does not exist" containerID="31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"
Oct 30 21:38:05.870656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.870612    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0"} err="failed to get container status \"31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0\": rpc error: code = NotFound desc = could not find container \"31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0\": container with ID starting with 31cffae3d825332725eeb8f616e2bbd8a77559ce68c3d3b51b0cb85c906095c0 not found: ID does not exist"
Oct 30 21:38:05.932991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.932917    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"]
Oct 30 21:38:05.938768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:05.938709    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl"]
Oct 30 21:38:06.139740 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:06.139698    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:38:06.140255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:06.139713    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:38:06.497701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:06.497611    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:07.028436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:07.028402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:07.161650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:07.161610    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:07.325975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:07.325936    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:07.365308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:07.365275    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" path="/var/lib/kubelet/pods/c4b73a33-2d90-4e3d-9680-022ad03c5a2e/volumes"
Oct 30 21:38:07.642568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:07.642483    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.809303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:08.809260    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.936432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:08.936400    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.298753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:10.298716    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.715169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:11.715126    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:12.677179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.677144    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:38:12.723427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.723397    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:38:12.724289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.724257    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz\" (UniqueName: \"kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz\") pod \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") "
Oct 30 21:38:12.724454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.724438    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume\") pod \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\" (UID: \"de45254d-bdba-4f6f-83de-2fa8c41dc00d\") "
Oct 30 21:38:12.724844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.724788    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume" pod "de45254d-bdba-4f6f-83de-2fa8c41dc00d" (UID: "de45254d-bdba-4f6f-83de-2fa8c41dc00d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:12.725088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.725072    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de45254d-bdba-4f6f-83de-2fa8c41dc00d-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:12.726865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.726795    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz" pod "de45254d-bdba-4f6f-83de-2fa8c41dc00d" (UID: "de45254d-bdba-4f6f-83de-2fa8c41dc00d"). InnerVolumeSpecName "kube-api-access-dsstz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:12.825991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.825955    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb\" (UniqueName: \"kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb\") pod \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") "
Oct 30 21:38:12.826179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.826004    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume\") pod \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\" (UID: \"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7\") "
Oct 30 21:38:12.826380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.826356    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume" pod "205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" (UID: "205e1b4d-a2c8-401c-bb17-a2af41b3f6b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:12.826646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.826632    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:12.826691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.826649    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsstz\" (UniqueName: \"kubernetes.io/projected/de45254d-bdba-4f6f-83de-2fa8c41dc00d-kube-api-access-dsstz\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:12.828345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.828320    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb" pod "205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" (UID: "205e1b4d-a2c8-401c-bb17-a2af41b3f6b7"). InnerVolumeSpecName "kube-api-access-nwhzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:12.927794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:12.927704    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwhzb\" (UniqueName: \"kubernetes.io/projected/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7-kube-api-access-nwhzb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:13.016799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.016757    4549 generic.go:356] "Generic (PLEG): container finished" podID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerID="3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6" exitCode=0
Oct 30 21:38:13.017050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.016817    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"
Oct 30 21:38:13.017050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.016811    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" event={"ID":"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7","Type":"ContainerDied","Data":"3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6"}
Oct 30 21:38:13.017050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.016966    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h" event={"ID":"205e1b4d-a2c8-401c-bb17-a2af41b3f6b7","Type":"ContainerDied","Data":"6c71cb50700cc985e830dafc23fff1eb638974f520f61fdb0e95b52e324e77ee"}
Oct 30 21:38:13.017050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.016998    4549 scope.go:122] "RemoveContainer" containerID="3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6"
Oct 30 21:38:13.018548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.018517    4549 generic.go:356] "Generic (PLEG): container finished" podID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerID="fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc" exitCode=0
Oct 30 21:38:13.018882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.018554    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" event={"ID":"de45254d-bdba-4f6f-83de-2fa8c41dc00d","Type":"ContainerDied","Data":"fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc"}
Oct 30 21:38:13.018882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.018610    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7" event={"ID":"de45254d-bdba-4f6f-83de-2fa8c41dc00d","Type":"ContainerDied","Data":"d2accb37294bc0c90a397762332541da7b9f03b94a7dba4d4111a7f4b21b5ef8"}
Oct 30 21:38:13.018882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.018562    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"
Oct 30 21:38:13.029289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.029231    4549 scope.go:122] "RemoveContainer" containerID="3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6"
Oct 30 21:38:13.029627 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:13.029605    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6\": container with ID starting with 3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6 not found: ID does not exist" containerID="3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6"
Oct 30 21:38:13.029692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.029637    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6"} err="failed to get container status \"3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6\": rpc error: code = NotFound desc = could not find container \"3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6\": container with ID starting with 3ab0673a2cd878afdf115e58b514db9dc5f517de919ede57286bc67e9f880cb6 not found: ID does not exist"
Oct 30 21:38:13.029692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.029658    4549 scope.go:122] "RemoveContainer" containerID="fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc"
Oct 30 21:38:13.041768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.041742    4549 scope.go:122] "RemoveContainer" containerID="fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc"
Oct 30 21:38:13.042139 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:13.042116    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc\": container with ID starting with fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc not found: ID does not exist" containerID="fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc"
Oct 30 21:38:13.042222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.042155    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc"} err="failed to get container status \"fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc\": rpc error: code = NotFound desc = could not find container \"fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc\": container with ID starting with fa76062c6747b47ec0e22c7e48341414897641b612ea6cbeb8a75e0e5f3b0abc not found: ID does not exist"
Oct 30 21:38:13.075268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.075232    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"]
Oct 30 21:38:13.080056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.080024    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h"]
Oct 30 21:38:13.116716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.116677    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"]
Oct 30 21:38:13.127789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.127751    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7"]
Oct 30 21:38:13.137202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.137170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:13.351701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.351660    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" path="/var/lib/kubelet/pods/205e1b4d-a2c8-401c-bb17-a2af41b3f6b7/volumes"
Oct 30 21:38:13.352448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.352418    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" path="/var/lib/kubelet/pods/de45254d-bdba-4f6f-83de-2fa8c41dc00d/volumes"
Oct 30 21:38:13.989486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:13.989449    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.117678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:14.117638    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.303478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:14.303272    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"]
Oct 30 21:38:14.303723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:14.303671    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" containerID="cri-o://af9b19bedcc88ba1ec93b28ef143475c5185aaeba604ed908e2e0f7c14b25b0d" gracePeriod=30
Oct 30 21:38:14.379285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:14.379247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.777699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:14.777660    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.939891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:14.933109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.299305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.299271    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"]
Oct 30 21:38:16.299883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.299484    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm" podUID="59ea4b8a-e525-418e-8976-94bb0892c61e" containerName="maas-api" containerID="cri-o://79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f" gracePeriod=30
Oct 30 21:38:16.417329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.417302    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.680921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.680894    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:38:16.754030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.736527    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls\") pod \"59ea4b8a-e525-418e-8976-94bb0892c61e\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") "
Oct 30 21:38:16.754030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.736603    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs\") pod \"59ea4b8a-e525-418e-8976-94bb0892c61e\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") "
Oct 30 21:38:16.754030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.736661    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v\" (UniqueName: \"kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v\") pod \"59ea4b8a-e525-418e-8976-94bb0892c61e\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") "
Oct 30 21:38:16.754030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.736767    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca\") pod \"59ea4b8a-e525-418e-8976-94bb0892c61e\" (UID: \"59ea4b8a-e525-418e-8976-94bb0892c61e\") "
Oct 30 21:38:16.754030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.750749    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca" pod "59ea4b8a-e525-418e-8976-94bb0892c61e" (UID: "59ea4b8a-e525-418e-8976-94bb0892c61e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:16.760257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.760215    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls" pod "59ea4b8a-e525-418e-8976-94bb0892c61e" (UID: "59ea4b8a-e525-418e-8976-94bb0892c61e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:16.786862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.786242    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs" pod "59ea4b8a-e525-418e-8976-94bb0892c61e" (UID: "59ea4b8a-e525-418e-8976-94bb0892c61e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:16.806866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.804084    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v" pod "59ea4b8a-e525-418e-8976-94bb0892c61e" (UID: "59ea4b8a-e525-418e-8976-94bb0892c61e"). InnerVolumeSpecName "kube-api-access-h9t9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:16.852885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.848882    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:16.852885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.848918    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ea4b8a-e525-418e-8976-94bb0892c61e-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:16.852885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.848933    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9t9v\" (UniqueName: \"kubernetes.io/projected/59ea4b8a-e525-418e-8976-94bb0892c61e-kube-api-access-h9t9v\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:16.852885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.848947    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ea4b8a-e525-418e-8976-94bb0892c61e-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:16.967531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:16.967449    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.126750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.126708    4549 generic.go:356] "Generic (PLEG): container finished" podID="59ea4b8a-e525-418e-8976-94bb0892c61e" containerID="79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f" exitCode=0
Oct 30 21:38:17.126974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.126764    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"
Oct 30 21:38:17.126974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.126791    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm" event={"ID":"59ea4b8a-e525-418e-8976-94bb0892c61e","Type":"ContainerDied","Data":"79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f"}
Oct 30 21:38:17.126974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.126865    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm" event={"ID":"59ea4b8a-e525-418e-8976-94bb0892c61e","Type":"ContainerDied","Data":"eb0fc9760b20e8e554681372b0762b5f3d29829371ea414c60a759005e3d41e8"}
Oct 30 21:38:17.126974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.126890    4549 scope.go:122] "RemoveContainer" containerID="79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f"
Oct 30 21:38:17.137238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.137213    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.138940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.138916    4549 scope.go:122] "RemoveContainer" containerID="79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f"
Oct 30 21:38:17.139272 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:17.139252    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f\": container with ID starting with 79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f not found: ID does not exist" containerID="79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f"
Oct 30 21:38:17.139350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.139281    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f"} err="failed to get container status \"79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f\": rpc error: code = NotFound desc = could not find container \"79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f\": container with ID starting with 79573f14b58a065a3dd69437b7e2297f957f8813bd715c7fe135e7382f03a21f not found: ID does not exist"
Oct 30 21:38:17.166092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.166054    4549 patch_prober.go:29] interesting pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:38:17.166271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.166111    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:38:17.187218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.187183    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"]
Oct 30 21:38:17.198006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.197974    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9477c-6966fc6c7b-bxrxm"]
Oct 30 21:38:17.327806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.327747    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.345856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.345810    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59ea4b8a-e525-418e-8976-94bb0892c61e" path="/var/lib/kubelet/pods/59ea4b8a-e525-418e-8976-94bb0892c61e/volumes"
Oct 30 21:38:17.575625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:17.575572    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.026871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.026789    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.622655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.622625    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:38:19.679179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.679040    4549 scope.go:122] "RemoveContainer" containerID="49ca274d2befbb500fc205c781f0e3ebad381a79abf5c4adbb83ed2f41067b3e"
Oct 30 21:38:19.690853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.690812    4549 scope.go:122] "RemoveContainer" containerID="af9b19bedcc88ba1ec93b28ef143475c5185aaeba604ed908e2e0f7c14b25b0d"
Oct 30 21:38:19.695016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.694981    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695041    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695128    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695195    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695234    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695381    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695412    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695464    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.695785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.695526    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data\") pod \"a88d883a-bb1c-4c36-8a95-6911ac25325d\" (UID: \"a88d883a-bb1c-4c36-8a95-6911ac25325d\") "
Oct 30 21:38:19.697094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.696505    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:19.697208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.696810    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:19.697208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.696903    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:19.697208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.696931    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:19.697208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.697059    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:19.699172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.699139    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:19.699532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.699505    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:19.699902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.699881    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:19.701007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.700984    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq" pod "a88d883a-bb1c-4c36-8a95-6911ac25325d" (UID: "a88d883a-bb1c-4c36-8a95-6911ac25325d"). InnerVolumeSpecName "kube-api-access-rgxjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:19.797880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797847    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.797880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797881    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797896    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797908    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797917    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797925    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a88d883a-bb1c-4c36-8a95-6911ac25325d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797936    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a88d883a-bb1c-4c36-8a95-6911ac25325d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797944    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a88d883a-bb1c-4c36-8a95-6911ac25325d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:19.798058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:19.797952    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgxjq\" (UniqueName: \"kubernetes.io/projected/a88d883a-bb1c-4c36-8a95-6911ac25325d-kube-api-access-rgxjq\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:20.149195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:20.149166    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"
Oct 30 21:38:20.149658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:20.149160    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" event={"ID":"a88d883a-bb1c-4c36-8a95-6911ac25325d","Type":"ContainerDied","Data":"af9b19bedcc88ba1ec93b28ef143475c5185aaeba604ed908e2e0f7c14b25b0d"}
Oct 30 21:38:20.149658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:20.149318    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7" event={"ID":"a88d883a-bb1c-4c36-8a95-6911ac25325d","Type":"ContainerDied","Data":"a2471cc72fd109ccd31f381c45453b8e29dc0e598e34d2b8160efa440c3d097e"}
Oct 30 21:38:20.240502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:20.240462    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"]
Oct 30 21:38:20.244606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:20.244577    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7"]
Oct 30 21:38:20.491100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:20.491013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.345550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:21.345519    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" path="/var/lib/kubelet/pods/a88d883a-bb1c-4c36-8a95-6911ac25325d/volumes"
Oct 30 21:38:22.013246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:22.013212    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:23.527570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:23.527531    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:23.948370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:23.948330    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:24.099033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.098989    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:24.222804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.221197    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"]
Oct 30 21:38:24.222804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.221429    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" containerID="cri-o://9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c" gracePeriod=30
Oct 30 21:38:24.241951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.241900    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"]
Oct 30 21:38:24.242193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.242157    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" containerID="cri-o://b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63" gracePeriod=30
Oct 30 21:38:24.244750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.244722    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"]
Oct 30 21:38:24.245002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.244973    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" containerID="cri-o://a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c" gracePeriod=30
Oct 30 21:38:24.290784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:24.290749    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.055945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.055876    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.182340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.182299    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.509317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.509263    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.656484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.656430    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.680855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.663104    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:38:25.680855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.663229    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:38:25.890176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:25.890142    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:26.576972 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:26.576930    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:38:26.630010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:26.629972    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:38:26.630210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:26.630034    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:38:26.746466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:26.746426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:27.583108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:27.583074    4549 patch_prober.go:29] interesting pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:38:27.583526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:27.583130    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:38:28.204141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:28.204093    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:28.818521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:28.818485    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:29.012083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.012044    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:29.551155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.551120    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:38:29.583686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.583661    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:38:29.587422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.587401    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:38:29.604696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.604671    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.604886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.604725    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.604886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.604754    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.604886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.604808    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.604886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.604861    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.605127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.604989    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.605127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605020    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.605127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605040    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.605279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605130    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.605279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605257    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket\") pod \"9306875b-38bb-450b-977e-e21d1a78c8c0\" (UID: \"9306875b-38bb-450b-977e-e21d1a78c8c0\") "
Oct 30 21:38:29.605387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605324    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.605787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605640    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.605787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605748    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.605991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605815    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:29.605991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605964    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.605991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605985    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9306875b-38bb-450b-977e-e21d1a78c8c0-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.606170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.605998    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.606170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.606010    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.606170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.606024    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.608528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.608500    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.608805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.608783    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:29.608947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.608912    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.609462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.609438    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg" pod "9306875b-38bb-450b-977e-e21d1a78c8c0" (UID: "9306875b-38bb-450b-977e-e21d1a78c8c0"). InnerVolumeSpecName "kube-api-access-pvxpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.707501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.707454    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.707957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.707932    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.707973    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708015    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708093    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708135    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708201    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708233    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708301    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708334    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708399    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708428    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708490    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708571    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708609    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708696    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708732    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs\") pod \"a13f1e31-b3d1-4707-bccf-f0fc82481748\" (UID: \"a13f1e31-b3d1-4707-bccf-f0fc82481748\") "
Oct 30 21:38:29.708885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.708771    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data\") pod \"44015348-bfc8-4a79-be03-7234b658ad3d\" (UID: \"44015348-bfc8-4a79-be03-7234b658ad3d\") "
Oct 30 21:38:29.709539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.709354    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.709539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.709377    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.709539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.709390    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.709539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.709405    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvxpg\" (UniqueName: \"kubernetes.io/projected/9306875b-38bb-450b-977e-e21d1a78c8c0-kube-api-access-pvxpg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.709624    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.709872    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710101    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710115    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710150    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710188    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710229    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710440    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710465    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:29.710948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.710689    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.713709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.713675    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:29.714029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.713987    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:29.714286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.714257    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.714400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.714299    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "kube-api-access-m44x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.715217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.715192    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.715330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.715284    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.716143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.716124    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb" pod "a13f1e31-b3d1-4707-bccf-f0fc82481748" (UID: "a13f1e31-b3d1-4707-bccf-f0fc82481748"). InnerVolumeSpecName "kube-api-access-cpgpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.717354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.717323    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy" pod "44015348-bfc8-4a79-be03-7234b658ad3d" (UID: "44015348-bfc8-4a79-be03-7234b658ad3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810767    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44015348-bfc8-4a79-be03-7234b658ad3d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810796    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810805    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810813    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810820    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m44x4\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-kube-api-access-m44x4\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810845    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810854    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.810859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810865    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810873    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810882    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810890    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810899    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810908    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpgpb\" (UniqueName: \"kubernetes.io/projected/a13f1e31-b3d1-4707-bccf-f0fc82481748-kube-api-access-cpgpb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810917    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13f1e31-b3d1-4707-bccf-f0fc82481748-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810925    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44015348-bfc8-4a79-be03-7234b658ad3d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810932    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13f1e31-b3d1-4707-bccf-f0fc82481748-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810938    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44015348-bfc8-4a79-be03-7234b658ad3d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:29.811232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:29.810945    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44015348-bfc8-4a79-be03-7234b658ad3d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:30.094936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.094763    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:30.225150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.225109    4549 generic.go:356] "Generic (PLEG): container finished" podID="44015348-bfc8-4a79-be03-7234b658ad3d" containerID="b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63" exitCode=0
Oct 30 21:38:30.225316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.225158    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"
Oct 30 21:38:30.225316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.225186    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" event={"ID":"44015348-bfc8-4a79-be03-7234b658ad3d","Type":"ContainerDied","Data":"b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63"}
Oct 30 21:38:30.225316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.225242    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk" event={"ID":"44015348-bfc8-4a79-be03-7234b658ad3d","Type":"ContainerDied","Data":"24a2be878e38190287cf0fb26454200e1fa233bb314f4fef6a6f5683c021fcb7"}
Oct 30 21:38:30.225316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.225272    4549 scope.go:122] "RemoveContainer" containerID="b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63"
Oct 30 21:38:30.227036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.227006    4549 generic.go:356] "Generic (PLEG): container finished" podID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerID="9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c" exitCode=0
Oct 30 21:38:30.227155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.227112    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" event={"ID":"9306875b-38bb-450b-977e-e21d1a78c8c0","Type":"ContainerDied","Data":"9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c"}
Oct 30 21:38:30.227227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.227160    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"
Oct 30 21:38:30.227227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.227179    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr" event={"ID":"9306875b-38bb-450b-977e-e21d1a78c8c0","Type":"ContainerDied","Data":"64fa4c9fe77cfda4329116ede9965a0e7e7b42a332508921071471af540f4c51"}
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.239529    4549 scope.go:122] "RemoveContainer" containerID="b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63"
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:30.240376    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63\": container with ID starting with b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63 not found: ID does not exist" containerID="b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63"
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.240417    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63"} err="failed to get container status \"b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63\": rpc error: code = NotFound desc = could not find container \"b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63\": container with ID starting with b17faab13c015ce8830220f881f607e8165eaafd80950b0c71fe1231e1c5ce63 not found: ID does not exist"
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.240442    4549 scope.go:122] "RemoveContainer" containerID="9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c"
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.240879    4549 generic.go:356] "Generic (PLEG): container finished" podID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerID="a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c" exitCode=0
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.241009    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" event={"ID":"a13f1e31-b3d1-4707-bccf-f0fc82481748","Type":"ContainerDied","Data":"a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c"}
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.241060    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56" event={"ID":"a13f1e31-b3d1-4707-bccf-f0fc82481748","Type":"ContainerDied","Data":"f39a765d43624e8288e80765475c649c3152baf9477e56b126b95e9dbdc4189d"}
Oct 30 21:38:30.242981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.241148    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"
Oct 30 21:38:30.273306 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.273286    4549 scope.go:122] "RemoveContainer" containerID="9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c"
Oct 30 21:38:30.273607 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:30.273584    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c\": container with ID starting with 9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c not found: ID does not exist" containerID="9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c"
Oct 30 21:38:30.273663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.273617    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c"} err="failed to get container status \"9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c\": rpc error: code = NotFound desc = could not find container \"9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c\": container with ID starting with 9ecf834607f1738ddc0047aeeff2bcc52554db7239b8e9748dcafaecaeeac29c not found: ID does not exist"
Oct 30 21:38:30.273663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.273635    4549 scope.go:122] "RemoveContainer" containerID="a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c"
Oct 30 21:38:30.284236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.284214    4549 scope.go:122] "RemoveContainer" containerID="a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c"
Oct 30 21:38:30.284614 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:30.284582    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c\": container with ID starting with a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c not found: ID does not exist" containerID="a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c"
Oct 30 21:38:30.284695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.284628    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c"} err="failed to get container status \"a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c\": rpc error: code = NotFound desc = could not find container \"a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c\": container with ID starting with a667560146f73932aeba92c45d9253e7940466abb9008738bd47d1658741c25c not found: ID does not exist"
Oct 30 21:38:30.296429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.296389    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"]
Oct 30 21:38:30.300715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.300689    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk"]
Oct 30 21:38:30.379224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.379126    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"]
Oct 30 21:38:30.383630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.383598    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr"]
Oct 30 21:38:30.421757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.421723    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"]
Oct 30 21:38:30.434096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.434060    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56"]
Oct 30 21:38:30.535742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.535693    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:30.677909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.677786    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:30.686361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.686331    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"]
Oct 30 21:38:30.687167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687146    4549 cpu_manager.go:414] "containerMap: removing container" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy"
Oct 30 21:38:30.687167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687167    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687181    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687188    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687199    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687204    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687217    4549 cpu_manager.go:414] "containerMap: removing container" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerName="payload-processing"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687222    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerName="payload-processing"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687230    4549 cpu_manager.go:414] "containerMap: removing container" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687235    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687240    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerName="main"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687244    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerName="main"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687252    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687256    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687264    4549 cpu_manager.go:414] "containerMap: removing container" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerName="payload-processing"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687268    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerName="payload-processing"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687277    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687281    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687287    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687291    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687298    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687302    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687309    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687314    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687321    4549 cpu_manager.go:414] "containerMap: removing container" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerName="payload-pre-processing"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687326    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerName="payload-pre-processing"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687338    4549 cpu_manager.go:414] "containerMap: removing container" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy"
Oct 30 21:38:30.687333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687343    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687350    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687355    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687367    4549 cpu_manager.go:414] "containerMap: removing container" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687372    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687379    4549 cpu_manager.go:414] "containerMap: removing container" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687383    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687393    4549 cpu_manager.go:414] "containerMap: removing container" podUID="59ea4b8a-e525-418e-8976-94bb0892c61e" containerName="maas-api"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687397    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59ea4b8a-e525-418e-8976-94bb0892c61e" containerName="maas-api"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687407    4549 cpu_manager.go:414] "containerMap: removing container" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687411    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687421    4549 cpu_manager.go:414] "containerMap: removing container" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687424    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687517    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687524    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ebd63a4-8699-4329-9480-3e022170dee4" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687530    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687537    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687543    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687553    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687561    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="4407ac7e-75a7-4990-9585-8d4f9502e5e0" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687572    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687579    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687589    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="537612db-c727-42c3-b8a7-febe5d6e1812" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687594    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="205e1b4d-a2c8-401c-bb17-a2af41b3f6b7" containerName="payload-pre-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687600    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="59ea4b8a-e525-418e-8976-94bb0892c61e" containerName="maas-api"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687606    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687611    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="59e347c6-5c61-4579-93c3-f6a576f6cb42" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687616    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="16e9ef29-3164-4072-9560-251a9db10da5" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687622    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d490321-2533-40fc-a4e8-a90521b435b8" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687628    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687635    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687640    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="de45254d-bdba-4f6f-83de-2fa8c41dc00d" containerName="payload-processing"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687645    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" containerName="istio-proxy"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687651    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a61de4a5-7c25-4ca5-9f38-ed1c4fcc0235" containerName="main"
Oct 30 21:38:30.688141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.687658    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="53b1a8bf-adc9-44a6-bcf0-510c985a735a" containerName="payload-processing"
Oct 30 21:38:30.692941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.692919    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:30.708417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.708387    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-66cc54\""
Oct 30 21:38:30.708596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.708570    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-66cc54-dockercfg-7589p\""
Oct 30 21:38:30.710887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.710865    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"]
Oct 30 21:38:30.984763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.984667    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljlcg\" (UniqueName: \"kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg\") pod \"payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:30.984763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:30.984741    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:31.085710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.085669    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:31.085973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.085953    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljlcg\" (UniqueName: \"kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg\") pod \"payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:31.086473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.086456    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:31.105273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.105241    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljlcg\" (UniqueName: \"kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg\") pod \"payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:31.307736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.307641    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:31.345363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.345327    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44015348-bfc8-4a79-be03-7234b658ad3d" path="/var/lib/kubelet/pods/44015348-bfc8-4a79-be03-7234b658ad3d/volumes"
Oct 30 21:38:31.345951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.345929    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9306875b-38bb-450b-977e-e21d1a78c8c0" path="/var/lib/kubelet/pods/9306875b-38bb-450b-977e-e21d1a78c8c0/volumes"
Oct 30 21:38:31.346478 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.346459    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a13f1e31-b3d1-4707-bccf-f0fc82481748" path="/var/lib/kubelet/pods/a13f1e31-b3d1-4707-bccf-f0fc82481748/volumes"
Oct 30 21:38:31.640362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.640329    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"]
Oct 30 21:38:31.641841 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:38:31.641802    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c405a84_a806_4b83_a9ed_6b2eafd74fb8.slice/crio-468fc0cff7471d04c10cf5ceb3ddfca2b1b21a223e0e88097033924a488a5a8f WatchSource:0}: Error finding container 468fc0cff7471d04c10cf5ceb3ddfca2b1b21a223e0e88097033924a488a5a8f: Status 404 returned error can't find the container with id 468fc0cff7471d04c10cf5ceb3ddfca2b1b21a223e0e88097033924a488a5a8f
Oct 30 21:38:31.807154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:31.807119    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:32.258985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:32.258939    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" event={"ID":"3c405a84-a806-4b83-a9ed-6b2eafd74fb8","Type":"ContainerStarted","Data":"05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c"}
Oct 30 21:38:32.259418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:32.259005    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" event={"ID":"3c405a84-a806-4b83-a9ed-6b2eafd74fb8","Type":"ContainerStarted","Data":"468fc0cff7471d04c10cf5ceb3ddfca2b1b21a223e0e88097033924a488a5a8f"}
Oct 30 21:38:32.259418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:32.259085    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:32.302392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:32.302334    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" podStartSLOduration=2.302319465 podStartE2EDuration="2.302319465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:32.299805441 +0000 UTC m=+2295.074205387" watchObservedRunningTime="2026-10-30 21:38:32.302319465 +0000 UTC m=+2295.076719448"
Oct 30 21:38:33.289589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:33.289543    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:33.419857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:33.415421    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:34.375167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:34.375127    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"]
Oct 30 21:38:34.403693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:34.403654    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:35.194767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:35.194605    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:36.684164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:36.684125    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:38.206814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:38.206753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:39.705693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:39.705657    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:40.449807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.449767    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"]
Oct 30 21:38:40.450858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450768    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy"
Oct 30 21:38:40.450858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450795    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6ccf846-20db-4a8d-874e-428b2a90d59d" containerName="istio-proxy"
Oct 30 21:38:40.450858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450846    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy"
Oct 30 21:38:40.450858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450854    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d79a5cf-d13a-4723-9ea2-517205e31827" containerName="istio-proxy"
Oct 30 21:38:40.451208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450871    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy"
Oct 30 21:38:40.451208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450880    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy"
Oct 30 21:38:40.451208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450947    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy"
Oct 30 21:38:40.451208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.450955    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5540263-fc2a-4fc6-b9c4-7045118690ee" containerName="istio-proxy"
Oct 30 21:38:40.451208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.451118    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b73a33-2d90-4e3d-9680-022ad03c5a2e" containerName="istio-proxy"
Oct 30 21:38:40.451208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.451131    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a88d883a-bb1c-4c36-8a95-6911ac25325d" containerName="istio-proxy"
Oct 30 21:38:40.455049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.455027    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.473455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.473424    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"]
Oct 30 21:38:40.473718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.473697    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.473792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.473752    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.473952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.473933    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.474014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.473965    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.474014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.473993    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nz5n\" (UniqueName: \"kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.474186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.474140    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.480015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.479980    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w6-95692f\"/\"auth-test-model-95692f-kserve-self-signed-certs\""
Oct 30 21:38:40.480158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.480075    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w6-95692f\"/\"openshift-service-ca.crt\""
Oct 30 21:38:40.481156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.481125    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w6-95692f\"/\"default-dockercfg-462bg\""
Oct 30 21:38:40.481275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.481213    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w6-95692f\"/\"kube-root-ca.crt\""
Oct 30 21:38:40.506666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.506633    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:40.575948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.575912    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576025    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576070    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576195    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nz5n\" (UniqueName: \"kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576322    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576368    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576377    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576478    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.576809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.576784    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.578599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.578573    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.578787 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.578768    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.600685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.600655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nz5n\" (UniqueName: \"kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n\") pod \"auth-test-model-95692f-kserve-76ffd8f685-fcmgs\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") " pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:40.770639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:40.770537    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:41.083950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:41.083897    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"]
Oct 30 21:38:41.084109 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:38:41.083983    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeece88f7_f624_4554_bcea_c1091bc0390f.slice/crio-71ae6d3745ad8a076479e5c10a8280df3b51e447a6b8beb4fcfa9557d9740bf7 WatchSource:0}: Error finding container 71ae6d3745ad8a076479e5c10a8280df3b51e447a6b8beb4fcfa9557d9740bf7: Status 404 returned error can't find the container with id 71ae6d3745ad8a076479e5c10a8280df3b51e447a6b8beb4fcfa9557d9740bf7
Oct 30 21:38:41.146557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:41.146522    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.413007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:41.412917    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs" event={"ID":"eece88f7-f624-4554-bcea-c1091bc0390f","Type":"ContainerStarted","Data":"8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac"}
Oct 30 21:38:41.413007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:41.412962    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs" event={"ID":"eece88f7-f624-4554-bcea-c1091bc0390f","Type":"ContainerStarted","Data":"71ae6d3745ad8a076479e5c10a8280df3b51e447a6b8beb4fcfa9557d9740bf7"}
Oct 30 21:38:41.456108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:41.456048    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs" podStartSLOduration=1.456029738 podStartE2EDuration="1.456029738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:41.45451913 +0000 UTC m=+2304.228919071" watchObservedRunningTime="2026-10-30 21:38:41.456029738 +0000 UTC m=+2304.230429679"
Oct 30 21:38:41.771422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:41.771328    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:42.604923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:42.604885    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.118620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:43.118583    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.259763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:43.259720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.268733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:43.268702    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:38:43.552629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:43.552544    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.827422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:43.827389    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:45.026687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:45.026651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:45.967188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:45.966646    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"]
Oct 30 21:38:46.041508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:46.041459    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:46.656770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:46.656725    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.330115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:48.330079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:49.404702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.404660    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" podUID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" containerName="main" containerID="cri-o://db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac" gracePeriod=45
Oct 30 21:38:49.687620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.687588    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:38:49.801305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801273    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp\" (UniqueName: \"kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp\") pod \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") "
Oct 30 21:38:49.801480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801369    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home\") pod \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") "
Oct 30 21:38:49.801480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801406    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache\") pod \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") "
Oct 30 21:38:49.801480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801445    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs\") pod \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") "
Oct 30 21:38:49.801672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801483    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir\") pod \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") "
Oct 30 21:38:49.801672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801516    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm\") pod \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\" (UID: \"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0\") "
Oct 30 21:38:49.801783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801721    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home" pod "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" (UID: "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:49.801783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801708    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache" pod "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" (UID: "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:49.801881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.801789    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir" pod "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" (UID: "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:49.802683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.802657    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:49.802840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.802687    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:49.802840 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.802701    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:49.804034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.804011    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs" pod "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" (UID: "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:49.804034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.804024    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp" pod "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" (UID: "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0"). InnerVolumeSpecName "kube-api-access-qzrgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:49.804162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.804062    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm" pod "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" (UID: "0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:49.904072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.904028    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzrgp\" (UniqueName: \"kubernetes.io/projected/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-kube-api-access-qzrgp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:49.904072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.904071    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:49.904290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:49.904085    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:50.174885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.171369    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.491212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.491121    4549 generic.go:356] "Generic (PLEG): container finished" podID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" containerID="db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac" exitCode=0
Oct 30 21:38:50.491212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.491190    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"
Oct 30 21:38:50.491705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.491210    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" event={"ID":"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0","Type":"ContainerDied","Data":"db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac"}
Oct 30 21:38:50.491705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.491247    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m" event={"ID":"0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0","Type":"ContainerDied","Data":"930225d4ca7b417b2b2d9820ae14afd308a9243ab7ff224def73844c2287f89c"}
Oct 30 21:38:50.491705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.491264    4549 scope.go:122] "RemoveContainer" containerID="db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac"
Oct 30 21:38:50.502337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.502320    4549 scope.go:122] "RemoveContainer" containerID="db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac"
Oct 30 21:38:50.502641 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:50.502617    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac\": container with ID starting with db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac not found: ID does not exist" containerID="db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac"
Oct 30 21:38:50.502683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.502652    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac"} err="failed to get container status \"db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac\": rpc error: code = NotFound desc = could not find container \"db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac\": container with ID starting with db1397df28dfdbb48019b5e10318cc73673d8cc9dfa97e9381ef43e91e0357ac not found: ID does not exist"
Oct 30 21:38:50.528058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.528029    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.544038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.544006    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"]
Oct 30 21:38:50.555130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.555102    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-67f8e0/test-model-67f8e0-kserve-57c44b8c4c-ndb8m"]
Oct 30 21:38:50.693656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.693621    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.772133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.772039    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:50.785460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.785425    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:50.827560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.827528    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"]
Oct 30 21:38:50.827805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.827777    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy" containerID="cri-o://dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac" gracePeriod=30
Oct 30 21:38:50.882233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:50.882202    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:51.344622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:51.344583    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" path="/var/lib/kubelet/pods/0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0/volumes"
Oct 30 21:38:51.510456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:51.510428    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:38:51.733196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:51.733112    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.192795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:53.192763    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:54.577235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:54.577201    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:56.109869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.109784    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:38:56.119952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.119917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:56.189027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.188979    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189037    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189118    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189147    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189266    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189318    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189340    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:56.189420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189361    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189488    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189551    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:56.189656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189589    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq\") pod \"87603398-cad5-4a8a-bea0-829f0d4acb2f\" (UID: \"87603398-cad5-4a8a-bea0-829f0d4acb2f\") "
Oct 30 21:38:56.189656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189636    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:56.189965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189861    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:56.189965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.189883    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:56.190447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.190420    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87603398-cad5-4a8a-bea0-829f0d4acb2f-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.190577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.190450    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.190577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.190464    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.190577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.190477    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.190577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.190490    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.194603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.194564    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:56.198275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.198249    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:38:56.203931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.203902    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "kube-api-access-j57bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:56.204052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.203963    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy" pod "87603398-cad5-4a8a-bea0-829f0d4acb2f" (UID: "87603398-cad5-4a8a-bea0-829f0d4acb2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:56.292463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.292421    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j57bq\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-kube-api-access-j57bq\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.292463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.292460    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.292679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.292475    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.292679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.292489    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87603398-cad5-4a8a-bea0-829f0d4acb2f-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:38:56.535456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.535367    4549 generic.go:356] "Generic (PLEG): container finished" podID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerID="dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac" exitCode=0
Oct 30 21:38:56.535456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.535415    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"
Oct 30 21:38:56.535696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.535450    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" event={"ID":"87603398-cad5-4a8a-bea0-829f0d4acb2f","Type":"ContainerDied","Data":"dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac"}
Oct 30 21:38:56.535696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.535517    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh" event={"ID":"87603398-cad5-4a8a-bea0-829f0d4acb2f","Type":"ContainerDied","Data":"7da0a7b3aab2d7d77e070ffb7eebbc2facff1fa86b6af81f588c4bf7399e35d7"}
Oct 30 21:38:56.535696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.535585    4549 scope.go:122] "RemoveContainer" containerID="dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac"
Oct 30 21:38:56.547528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.547511    4549 scope.go:122] "RemoveContainer" containerID="dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac"
Oct 30 21:38:56.547779 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:56.547757    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac\": container with ID starting with dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac not found: ID does not exist" containerID="dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac"
Oct 30 21:38:56.547881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.547785    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac"} err="failed to get container status \"dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac\": rpc error: code = NotFound desc = could not find container \"dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac\": container with ID starting with dfa36e5e83fe0ece13681b80534d7ec162727737fbc2019b60335195aed188ac not found: ID does not exist"
Oct 30 21:38:56.578792 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:38:56.578758    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:38:56.587622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.587598    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"]
Oct 30 21:38:56.591402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.591380    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh"]
Oct 30 21:38:56.675415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.675382    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"]
Oct 30 21:38:56.841158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.841120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:56.959565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:56.959532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:57.385881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:57.383436    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" path="/var/lib/kubelet/pods/87603398-cad5-4a8a-bea0-829f0d4acb2f/volumes"
Oct 30 21:38:57.694074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:57.693989    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:59.166469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.166437    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"]
Oct 30 21:38:59.167119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.167104    4549 cpu_manager.go:414] "containerMap: removing container" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy"
Oct 30 21:38:59.167170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.167120    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy"
Oct 30 21:38:59.167170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.167133    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" containerName="main"
Oct 30 21:38:59.167170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.167138    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" containerName="main"
Oct 30 21:38:59.167269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.167248    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a15e9fd-6ae8-4f1e-9b5c-cc16cbc77cc0" containerName="main"
Oct 30 21:38:59.167269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.167258    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="87603398-cad5-4a8a-bea0-829f0d4acb2f" containerName="istio-proxy"
Oct 30 21:38:59.172430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.172407    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.190495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.190461    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"]
Oct 30 21:38:59.194545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.194514    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-66cc54-serving-cert\""
Oct 30 21:38:59.285283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.285251    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:59.345886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.345803    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4plgf\" (UniqueName: \"kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.346208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.346182    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.346360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.346345    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.346495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.346463    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.406179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.406139    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:59.448507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.448421    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4plgf\" (UniqueName: \"kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.448507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.448476    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.448721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.448606    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.448721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.448642    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.449296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.449272    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.451160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.451141    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.451266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.451196    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.467943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.467914    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4plgf\" (UniqueName: \"kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf\") pod \"maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.488096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.488058    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:38:59.532166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.532130    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:59.797390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.797363    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"]
Oct 30 21:38:59.798744 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:38:59.798713    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b1f981_df5b_4f82_8330_be68205dd0b4.slice/crio-11fd09b4658bcb23175ed0ec7a7a10ca071e9d770c9e19916c2aca49f902acb4 WatchSource:0}: Error finding container 11fd09b4658bcb23175ed0ec7a7a10ca071e9d770c9e19916c2aca49f902acb4: Status 404 returned error can't find the container with id 11fd09b4658bcb23175ed0ec7a7a10ca071e9d770c9e19916c2aca49f902acb4
Oct 30 21:38:59.800160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:38:59.800143    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:39:00.582499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:00.582460    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw" event={"ID":"92b1f981-df5b-4f82-8330-be68205dd0b4","Type":"ContainerStarted","Data":"d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6"}
Oct 30 21:39:00.583084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:00.582533    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:39:00.583084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:00.582560    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw" event={"ID":"92b1f981-df5b-4f82-8330-be68205dd0b4","Type":"ContainerStarted","Data":"11fd09b4658bcb23175ed0ec7a7a10ca071e9d770c9e19916c2aca49f902acb4"}
Oct 30 21:39:00.624621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:00.624566    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw" podStartSLOduration=1.324731221 podStartE2EDuration="1.62455023s" totalImagesPullingTime="299.819009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:00.620749602 +0000 UTC m=+2323.395149542" watchObservedRunningTime="2026-10-30 21:39:00.62455023 +0000 UTC m=+2323.398950168"
Oct 30 21:39:00.916824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:00.916739    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:00.997276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:00.997232    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx" podUID="de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" containerName="main" containerID="cri-o://dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199" gracePeriod=45
Oct 30 21:39:01.280812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.280786    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:39:01.382540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382506    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs\" (UniqueName: \"kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs\") pod \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") "
Oct 30 21:39:01.382714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382576    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir\") pod \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") "
Oct 30 21:39:01.382852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382812    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache\") pod \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") "
Oct 30 21:39:01.382907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382881    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir" pod "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" (UID: "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:01.382960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382910    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm\") pod \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") "
Oct 30 21:39:01.383007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382958    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home\") pod \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") "
Oct 30 21:39:01.383007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.382991    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz\" (UniqueName: \"kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz\") pod \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\" (UID: \"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4\") "
Oct 30 21:39:01.383146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.383074    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache" pod "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" (UID: "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:01.383282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.383259    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home" pod "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" (UID: "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:01.383513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.383495    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:01.383607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.383514    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:01.383607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.383525    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:01.385171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.385144    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm" pod "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" (UID: "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:01.385278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.385183    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs" pod "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" (UID: "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:39:01.385278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.385253    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz" pod "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" (UID: "de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4"). InnerVolumeSpecName "kube-api-access-bm8cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:01.485161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.485068    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:01.485161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.485102    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm8cz\" (UniqueName: \"kubernetes.io/projected/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-kube-api-access-bm8cz\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:01.485161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.485116    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:01.591011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.590978    4549 generic.go:356] "Generic (PLEG): container finished" podID="de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" containerID="dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199" exitCode=0
Oct 30 21:39:01.591481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.591026    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"
Oct 30 21:39:01.591481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.591069    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx" event={"ID":"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4","Type":"ContainerDied","Data":"dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199"}
Oct 30 21:39:01.591481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.591112    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx" event={"ID":"de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4","Type":"ContainerDied","Data":"1a624c5fba15092790cb7b242218dac33a253d2b79aa8b44b547e4fd1d99a902"}
Oct 30 21:39:01.591481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.591132    4549 scope.go:122] "RemoveContainer" containerID="dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199"
Oct 30 21:39:01.602315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.602292    4549 scope.go:122] "RemoveContainer" containerID="dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199"
Oct 30 21:39:01.602578 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:01.602559    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199\": container with ID starting with dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199 not found: ID does not exist" containerID="dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199"
Oct 30 21:39:01.602632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.602588    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199"} err="failed to get container status \"dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199\": rpc error: code = NotFound desc = could not find container \"dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199\": container with ID starting with dfefadf527e8c0bea0c67866d97835b626adb9ef330c6acba7fafb7e75648199 not found: ID does not exist"
Oct 30 21:39:01.642686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.642648    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"]
Oct 30 21:39:01.658607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.658561    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f23eae/test-model-f23eae-kserve-78bc8dbb7-sjhlx"]
Oct 30 21:39:01.676293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:01.676233    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerName="payload-processing" containerID="cri-o://bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4" gracePeriod=25
Oct 30 21:39:02.406439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:02.406405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:03.345191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:03.345153    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" path="/var/lib/kubelet/pods/de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4/volumes"
Oct 30 21:39:03.796871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:03.796727    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:05.193622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:05.193588    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:05.685123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:05.685084    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:39:06.597579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:06.597548    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:39:06.696481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:06.696439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:08.198552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:08.198520    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:08.678421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:08.678390    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.011512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.011423    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.149352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.149316    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.535963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.535932    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"]
Oct 30 21:39:09.537004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.536977    4549 cpu_manager.go:414] "containerMap: removing container" podUID="de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" containerName="main"
Oct 30 21:39:09.537004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.537004    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" containerName="main"
Oct 30 21:39:09.537236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.537220    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="de2fc55a-f5dd-481a-99f4-a6d04c9c9cf4" containerName="main"
Oct 30 21:39:09.547957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.547930    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.564959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.564929    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"]
Oct 30 21:39:09.618620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.618586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.629047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.629012    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-889195-openshift-default-dockercfg-l5cnq\""
Oct 30 21:39:09.633824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.633796    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.633963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.633913    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634000    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634042    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634073    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634188    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634277    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khqhf\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634308    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.634451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.634438    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735238    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735288    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735308    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735340    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735378    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735431    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735509    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khqhf\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735546    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735696    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.735930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.735746    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.736244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.736094    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.736244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.736197    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.736411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.736329    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.736555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.736530    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.737928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.737906    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.738216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.738195    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.757685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.757653    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.757939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.757920    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khqhf\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf\") pod \"e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") " pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:09.761232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.761203    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.867853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:09.864816    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:10.010097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.010058    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:10.221553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.221520    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"]
Oct 30 21:39:10.223170 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:39:10.223145    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5bc90d_b046_4a1f_88de_93bbb58aac27.slice/crio-ed1e45e51e953a86b3228a7f725e26cc73bfc413fd72bcff36231ba5bf9e1298 WatchSource:0}: Error finding container ed1e45e51e953a86b3228a7f725e26cc73bfc413fd72bcff36231ba5bf9e1298: Status 404 returned error can't find the container with id ed1e45e51e953a86b3228a7f725e26cc73bfc413fd72bcff36231ba5bf9e1298
Oct 30 21:39:10.225958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.225924    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:39:10.226042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.225993    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:39:10.226042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.226025    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:39:10.661365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.661317    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" event={"ID":"2c5bc90d-b046-4a1f-88de-93bbb58aac27","Type":"ContainerStarted","Data":"e6981a78c4ffd977f09c2c5f95ba7e73bbe9767e37e9067f56f4ff6ee33bbd30"}
Oct 30 21:39:10.661808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.661395    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" event={"ID":"2c5bc90d-b046-4a1f-88de-93bbb58aac27","Type":"ContainerStarted","Data":"ed1e45e51e953a86b3228a7f725e26cc73bfc413fd72bcff36231ba5bf9e1298"}
Oct 30 21:39:10.719912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.719862    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podStartSLOduration=1.719848171 podStartE2EDuration="1.719848171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:10.715272343 +0000 UTC m=+2333.489672284" watchObservedRunningTime="2026-10-30 21:39:10.719848171 +0000 UTC m=+2333.494248104"
Oct 30 21:39:10.888794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.879967    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:10.987519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:10.983530    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"]
Oct 30 21:39:11.020398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.017342    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.020398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.019348    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"]
Oct 30 21:39:11.038861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.034043    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"openshift-service-ca.crt\""
Oct 30 21:39:11.038861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.034240    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"kube-root-ca.crt\""
Oct 30 21:39:11.054281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.054241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:11.058248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.058158    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.058424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.058298    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.058424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.058348    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.058424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.058402    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.058595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.058424    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.058595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.058460    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhlx8\" (UniqueName: \"kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.082113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.082082    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"default-dockercfg-wz7zg\""
Oct 30 21:39:11.082298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.082113    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"auth-test-model-70d3a9-kserve-self-signed-certs\""
Oct 30 21:39:11.160127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160094    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160144    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160175    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160208    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160225    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160245    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhlx8\" (UniqueName: \"kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160701    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.160795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.160715    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.161247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.161205    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.163260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.163230    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.164537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.164494    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.181563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.181536    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhlx8\" (UniqueName: \"kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8\") pod \"auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.346858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.344985    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:11.503860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.501970    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:11.677714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.677680    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"]
Oct 30 21:39:11.678864 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:39:11.678801    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc9f9e96_394b_425c_affa_e3a27d20f77f.slice/crio-2aa9c5e59ec829eafb7be089ee763c92e0adaf6ac64dbab1c5be771e85bd66b4 WatchSource:0}: Error finding container 2aa9c5e59ec829eafb7be089ee763c92e0adaf6ac64dbab1c5be771e85bd66b4: Status 404 returned error can't find the container with id 2aa9c5e59ec829eafb7be089ee763c92e0adaf6ac64dbab1c5be771e85bd66b4
Oct 30 21:39:11.872879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:11.872803    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:12.680273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:12.680226    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" event={"ID":"fc9f9e96-394b-425c-affa-e3a27d20f77f","Type":"ContainerStarted","Data":"dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda"}
Oct 30 21:39:12.680720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:12.680285    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:12.680720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:12.680314    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" event={"ID":"fc9f9e96-394b-425c-affa-e3a27d20f77f","Type":"ContainerStarted","Data":"2aa9c5e59ec829eafb7be089ee763c92e0adaf6ac64dbab1c5be771e85bd66b4"}
Oct 30 21:39:12.682022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:12.681992    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:39:12.723938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:12.723874    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" podStartSLOduration=2.723851828 podStartE2EDuration="2.723851828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:12.720046013 +0000 UTC m=+2335.494445952" watchObservedRunningTime="2026-10-30 21:39:12.723851828 +0000 UTC m=+2335.498251763"
Oct 30 21:39:13.210205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:13.210164    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:13.353345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:13.353307    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:13.399855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:13.386520    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:14.898209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:14.898167    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:15.098946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:15.098907    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:15.591970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:15.591927    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:15.684869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:15.684798    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:39:16.858327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:16.858292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.395285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:17.395246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.536114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:17.536079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.684388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:17.684287    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:18.029575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:18.029478    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:19.126481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:19.126437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:20.328991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:20.328957    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:20.909388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:20.909351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:21.352364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:21.352332    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:21.367254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:21.367226    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:21.751903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:21.751865    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:21.979733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:21.979704    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:39:22.065581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.065549    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume\") pod \"e4287f93-d448-4133-a8bd-aec42fb593fd\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") "
Oct 30 21:39:22.065768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.065599    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl\" (UniqueName: \"kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl\") pod \"e4287f93-d448-4133-a8bd-aec42fb593fd\" (UID: \"e4287f93-d448-4133-a8bd-aec42fb593fd\") "
Oct 30 21:39:22.066034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.065995    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume" pod "e4287f93-d448-4133-a8bd-aec42fb593fd" (UID: "e4287f93-d448-4133-a8bd-aec42fb593fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:39:22.066307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.066287    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4287f93-d448-4133-a8bd-aec42fb593fd-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:22.068209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.068186    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl" pod "e4287f93-d448-4133-a8bd-aec42fb593fd" (UID: "e4287f93-d448-4133-a8bd-aec42fb593fd"). InnerVolumeSpecName "kube-api-access-tswwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:22.167330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.167288    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tswwl\" (UniqueName: \"kubernetes.io/projected/e4287f93-d448-4133-a8bd-aec42fb593fd-kube-api-access-tswwl\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:22.407020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.406990    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.747431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.747330    4549 generic.go:356] "Generic (PLEG): container finished" podID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerID="bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4" exitCode=0
Oct 30 21:39:22.747431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.747386    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"
Oct 30 21:39:22.747431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.747411    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" event={"ID":"e4287f93-d448-4133-a8bd-aec42fb593fd","Type":"ContainerDied","Data":"bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4"}
Oct 30 21:39:22.747728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.747466    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg" event={"ID":"e4287f93-d448-4133-a8bd-aec42fb593fd","Type":"ContainerDied","Data":"fe2f6da722ab317d5777b3384faf6f16869efdcdd0cda88d754c0f8ae4bb1660"}
Oct 30 21:39:22.747728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.747488    4549 scope.go:122] "RemoveContainer" containerID="bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4"
Oct 30 21:39:22.758926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.758907    4549 scope.go:122] "RemoveContainer" containerID="bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4"
Oct 30 21:39:22.759228 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:22.759201    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4\": container with ID starting with bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4 not found: ID does not exist" containerID="bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4"
Oct 30 21:39:22.759332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.759235    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4"} err="failed to get container status \"bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4\": rpc error: code = NotFound desc = could not find container \"bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4\": container with ID starting with bb1f3a2dbb7b5ce0597dfee778a93324cab40902d47f19161314e50cef7567e4 not found: ID does not exist"
Oct 30 21:39:22.807497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.807459    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"]
Oct 30 21:39:22.816792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:22.816751    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg"]
Oct 30 21:39:23.350352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:23.350320    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" path="/var/lib/kubelet/pods/e4287f93-d448-4133-a8bd-aec42fb593fd/volumes"
Oct 30 21:39:23.867979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:23.867936    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.531082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:24.531047    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.690948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:24.690899    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.840709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:24.840673    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"]
Oct 30 21:39:24.841028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:24.840994    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" containerID="cri-o://2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365" gracePeriod=30
Oct 30 21:39:24.896675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:24.896631    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:25.427992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:25.427956    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:26.577846 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:26.577791    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:39:27.020691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:27.020601    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:27.270567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:27.270523    4549 patch_prober.go:29] interesting pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:39:27.270770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:27.270610    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:39:28.512088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:28.512049    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:30.071271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.071233    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:30.152617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.152583    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:39:30.316710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.316678    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.316994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.316782    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.316994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.316816    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.316994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.316888    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.316994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.316927    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.316994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.316955    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.317251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317092    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.317251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317138    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.317251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317184    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket\") pod \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\" (UID: \"b8c7a62d-0ac8-4dec-97ab-a821fa976299\") "
Oct 30 21:39:30.317415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317379    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:30.317471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317440    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:30.317675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317632    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:39:30.317675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317662    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:30.317890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317721    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:30.318013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.317991    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.318102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.318021    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.318102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.318039    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.318102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.318054    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.318102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.318067    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.319340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.319317    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:39:30.319615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.319598    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:30.319990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.319968    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:30.320074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.320035    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r" pod "b8c7a62d-0ac8-4dec-97ab-a821fa976299" (UID: "b8c7a62d-0ac8-4dec-97ab-a821fa976299"). InnerVolumeSpecName "kube-api-access-65n2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:30.419784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.419687    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.419784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.419727    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65n2r\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-kube-api-access-65n2r\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.419784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.419738    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.419784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.419750    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8c7a62d-0ac8-4dec-97ab-a821fa976299-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:30.804719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.804629    4549 generic.go:356] "Generic (PLEG): container finished" podID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerID="2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365" exitCode=0
Oct 30 21:39:30.804719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.804687    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"
Oct 30 21:39:30.804952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.804713    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" event={"ID":"b8c7a62d-0ac8-4dec-97ab-a821fa976299","Type":"ContainerDied","Data":"2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365"}
Oct 30 21:39:30.804952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.804779    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7" event={"ID":"b8c7a62d-0ac8-4dec-97ab-a821fa976299","Type":"ContainerDied","Data":"a025b97ab6f51ea57c88b0d60769e4890f4ed5932c0f9f526cf5d13970dc1b2f"}
Oct 30 21:39:30.804952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.804808    4549 scope.go:122] "RemoveContainer" containerID="2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365"
Oct 30 21:39:30.817533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.817517    4549 scope.go:122] "RemoveContainer" containerID="77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"
Oct 30 21:39:30.827728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.827699    4549 scope.go:122] "RemoveContainer" containerID="2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365"
Oct 30 21:39:30.828102 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:30.828082    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365\": container with ID starting with 2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365 not found: ID does not exist" containerID="2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365"
Oct 30 21:39:30.828173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.828115    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365"} err="failed to get container status \"2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365\": rpc error: code = NotFound desc = could not find container \"2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365\": container with ID starting with 2d06a69c2d37bb352f34d01a0d088ca410175cc7da36d53242d7f5aa993c5365 not found: ID does not exist"
Oct 30 21:39:30.828173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.828134    4549 scope.go:122] "RemoveContainer" containerID="77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"
Oct 30 21:39:30.828408 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:30.828384    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899\": container with ID starting with 77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899 not found: ID does not exist" containerID="77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"
Oct 30 21:39:30.828473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.828416    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899"} err="failed to get container status \"77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899\": rpc error: code = NotFound desc = could not find container \"77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899\": container with ID starting with 77746849056ca6c488d034b3a653312469daac277798a6ee33a1090a8d9d0899 not found: ID does not exist"
Oct 30 21:39:30.858981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.858949    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"]
Oct 30 21:39:30.869483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:30.869455    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7"]
Oct 30 21:39:31.348510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:31.348467    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" path="/var/lib/kubelet/pods/b8c7a62d-0ac8-4dec-97ab-a821fa976299/volumes"
Oct 30 21:39:31.805547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:31.805446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:33.592126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:33.586890    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:35.025142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:35.025102    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:35.169284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:35.169247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:35.465024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:35.464982    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.013879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:36.013787    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.163360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:36.163319    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.319128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:36.319100    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.456213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:36.456174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.841443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:36.841402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:38.317474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:38.317437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:39.799664 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:39.799611    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:39.950169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:39.950120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:40.100141 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:40.100110    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:40.465488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:40.465396    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"]
Oct 30 21:39:40.538191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:40.538158    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:41.799166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:41.799127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:42.084346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.084314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:42.175282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.175250    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"]
Oct 30 21:39:42.176028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.175998    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerName="payload-processing"
Oct 30 21:39:42.176028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176022    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerName="payload-processing"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176055    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176060    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176080    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176085    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176202    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176213    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8c7a62d-0ac8-4dec-97ab-a821fa976299" containerName="istio-proxy"
Oct 30 21:39:42.176250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.176225    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4287f93-d448-4133-a8bd-aec42fb593fd" containerName="payload-processing"
Oct 30 21:39:42.181916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.181892    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.197308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.197281    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"]
Oct 30 21:39:42.204654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.204624    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-9cd83f-dockercfg-h7ssb\""
Oct 30 21:39:42.224733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.224692    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:42.277802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.277761    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-9cd83f\""
Oct 30 21:39:42.360138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.359931    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm8p2\" (UniqueName: \"kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2\") pod \"payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.364850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.360537    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.467324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.467277    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm8p2\" (UniqueName: \"kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2\") pod \"payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.467595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.467448    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.468362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.468339    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.486429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.486396    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm8p2\" (UniqueName: \"kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2\") pod \"payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.500376 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.500350    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:42.804160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.804130    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"]
Oct 30 21:39:42.805243 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:39:42.805210    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2a12c4_4541_45a6_b3ca_913cc45d0491.slice/crio-222237e26374ba1220355ec99bdac9a7134fd57105cc6411bca7ce8e5f834d9f WatchSource:0}: Error finding container 222237e26374ba1220355ec99bdac9a7134fd57105cc6411bca7ce8e5f834d9f: Status 404 returned error can't find the container with id 222237e26374ba1220355ec99bdac9a7134fd57105cc6411bca7ce8e5f834d9f
Oct 30 21:39:42.890302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:42.890266    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" event={"ID":"9c2a12c4-4541-45a6-b3ca-913cc45d0491","Type":"ContainerStarted","Data":"222237e26374ba1220355ec99bdac9a7134fd57105cc6411bca7ce8e5f834d9f"}
Oct 30 21:39:43.279268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:43.279237    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:43.898990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:43.898945    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" event={"ID":"9c2a12c4-4541-45a6-b3ca-913cc45d0491","Type":"ContainerStarted","Data":"361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc"}
Oct 30 21:39:43.899438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:43.899126    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:43.941097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:43.941051    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" podStartSLOduration=1.941038583 podStartE2EDuration="1.941038583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:43.940938714 +0000 UTC m=+2366.715338665" watchObservedRunningTime="2026-10-30 21:39:43.941038583 +0000 UTC m=+2366.715438522"
Oct 30 21:39:44.772374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:44.772341    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:44.911120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:44.911080    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:45.034067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:45.033979    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:46.402960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:46.402928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:47.878676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:47.878636    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:49.320446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:49.320405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:50.811950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:50.811915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:52.008100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:52.008060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:52.532952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:52.532911    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:52.662876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:52.662843    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:52.814349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:52.814251    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:53.167357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:53.167319    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:54.324271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:54.324241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:54.908404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:54.908376    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:39:55.489029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.488972    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" podUID="fc9f9e96-394b-425c-affa-e3a27d20f77f" containerName="main" containerID="cri-o://dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda" gracePeriod=45
Oct 30 21:39:55.762055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.762028    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:55.827884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.827482    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm\") pod \"fc9f9e96-394b-425c-affa-e3a27d20f77f\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") "
Oct 30 21:39:55.827884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.827597    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir\") pod \"fc9f9e96-394b-425c-affa-e3a27d20f77f\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") "
Oct 30 21:39:55.827884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.827648    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs\") pod \"fc9f9e96-394b-425c-affa-e3a27d20f77f\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") "
Oct 30 21:39:55.827884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.827757    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache\") pod \"fc9f9e96-394b-425c-affa-e3a27d20f77f\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") "
Oct 30 21:39:55.827884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.827880    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8\" (UniqueName: \"kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8\") pod \"fc9f9e96-394b-425c-affa-e3a27d20f77f\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") "
Oct 30 21:39:55.828497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.827907    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home\") pod \"fc9f9e96-394b-425c-affa-e3a27d20f77f\" (UID: \"fc9f9e96-394b-425c-affa-e3a27d20f77f\") "
Oct 30 21:39:55.828497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.828163    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir" pod "fc9f9e96-394b-425c-affa-e3a27d20f77f" (UID: "fc9f9e96-394b-425c-affa-e3a27d20f77f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:55.828497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.828402    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home" pod "fc9f9e96-394b-425c-affa-e3a27d20f77f" (UID: "fc9f9e96-394b-425c-affa-e3a27d20f77f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:55.828661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.828511    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:55.828661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.828534    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:55.829316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.829183    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache" pod "fc9f9e96-394b-425c-affa-e3a27d20f77f" (UID: "fc9f9e96-394b-425c-affa-e3a27d20f77f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:55.831130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.831100    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs" pod "fc9f9e96-394b-425c-affa-e3a27d20f77f" (UID: "fc9f9e96-394b-425c-affa-e3a27d20f77f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:39:55.831440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.831417    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8" pod "fc9f9e96-394b-425c-affa-e3a27d20f77f" (UID: "fc9f9e96-394b-425c-affa-e3a27d20f77f"). InnerVolumeSpecName "kube-api-access-mhlx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:55.831878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.831847    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm" pod "fc9f9e96-394b-425c-affa-e3a27d20f77f" (UID: "fc9f9e96-394b-425c-affa-e3a27d20f77f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:39:55.930080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.930044    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:55.930080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.930074    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhlx8\" (UniqueName: \"kubernetes.io/projected/fc9f9e96-394b-425c-affa-e3a27d20f77f-kube-api-access-mhlx8\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:55.930080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.930083    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc9f9e96-394b-425c-affa-e3a27d20f77f-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:55.930330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.930093    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc9f9e96-394b-425c-affa-e3a27d20f77f-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:39:55.939147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.939109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:55.996604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.996516    4549 generic.go:356] "Generic (PLEG): container finished" podID="fc9f9e96-394b-425c-affa-e3a27d20f77f" containerID="dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda" exitCode=0
Oct 30 21:39:55.996604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.996566    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"
Oct 30 21:39:55.996802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.996597    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" event={"ID":"fc9f9e96-394b-425c-affa-e3a27d20f77f","Type":"ContainerDied","Data":"dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda"}
Oct 30 21:39:55.996802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.996649    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" event={"ID":"fc9f9e96-394b-425c-affa-e3a27d20f77f","Type":"ContainerDied","Data":"2aa9c5e59ec829eafb7be089ee763c92e0adaf6ac64dbab1c5be771e85bd66b4"}
Oct 30 21:39:55.996802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:55.996667    4549 scope.go:122] "RemoveContainer" containerID="dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda"
Oct 30 21:39:56.007976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:56.007812    4549 scope.go:122] "RemoveContainer" containerID="dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda"
Oct 30 21:39:56.008175 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:56.008151    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda\": container with ID starting with dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda not found: ID does not exist" containerID="dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda"
Oct 30 21:39:56.008227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:56.008190    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda"} err="failed to get container status \"dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda\": rpc error: code = NotFound desc = could not find container \"dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda\": container with ID starting with dfb866598ce008a717d73e3b27e62c33f1567a386352b01b0c4ec47e5b459cda not found: ID does not exist"
Oct 30 21:39:56.047726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:56.047676    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"]
Oct 30 21:39:56.052040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:56.052009    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2"]
Oct 30 21:39:56.575210 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:39:56.575172    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:39:56.741343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:56.741301    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-70d3a9/auth-test-model-70d3a9-kserve-8cd685b5c-9cfr2" podUID="fc9f9e96-394b-425c-affa-e3a27d20f77f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.179:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:39:57.345570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:57.345537    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc9f9e96-394b-425c-affa-e3a27d20f77f" path="/var/lib/kubelet/pods/fc9f9e96-394b-425c-affa-e3a27d20f77f/volumes"
Oct 30 21:39:57.348702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:57.348671    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:58.732026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:58.731951    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.122639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:59.122599    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.287891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:59.287856    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.421161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:59.421056    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"]
Oct 30 21:39:59.421364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:59.421339    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy" containerID="cri-o://a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855" gracePeriod=30
Oct 30 21:39:59.476200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:39:59.476163    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:00.300378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.300336    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:00.443371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.443335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:00.722650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.722617    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"]
Oct 30 21:40:00.723412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.723387    4549 cpu_manager.go:414] "containerMap: removing container" podUID="fc9f9e96-394b-425c-affa-e3a27d20f77f" containerName="main"
Oct 30 21:40:00.723412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.723408    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc9f9e96-394b-425c-affa-e3a27d20f77f" containerName="main"
Oct 30 21:40:00.723613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.723529    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc9f9e96-394b-425c-affa-e3a27d20f77f" containerName="main"
Oct 30 21:40:00.728649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.728623    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.759564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.759528    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-55c5e0-openshift-default-dockercfg-vt8nw\""
Oct 30 21:40:00.762012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.761980    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"]
Oct 30 21:40:00.815175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.815138    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:00.840926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.840884    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.840926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.840931    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkkgn\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841029    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841136    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841256    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841376    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841434    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841495    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.841554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.841519    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.959661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.953730    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.960209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.954586    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.960452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.960247    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.960648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.960621    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.960813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.960795    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.961406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.961384    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.961594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.961577    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.961797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.961780    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.961982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.961964    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.962112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.962092    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkkgn\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.972725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.972641    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.973071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.969467    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.973217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.969690    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:00.973598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:00.969142    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:01.004917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.003795    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkkgn\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:01.008621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.008552    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:01.008621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.008614    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:01.012867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.011361    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token\") pod \"e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") " pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:01.072150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.072112    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:01.311309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.311201    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:01.468987 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.468960    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"]
Oct 30 21:40:01.470911 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:40:01.470878    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c9470c_c9cb_41f2_be69_23b1685f1191.slice/crio-f98689de6c6cddb9692fa31904d7f15bd8073021fa246e5daa33ab04584293c8 WatchSource:0}: Error finding container f98689de6c6cddb9692fa31904d7f15bd8073021fa246e5daa33ab04584293c8: Status 404 returned error can't find the container with id f98689de6c6cddb9692fa31904d7f15bd8073021fa246e5daa33ab04584293c8
Oct 30 21:40:01.474573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.474529    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:40:01.474704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.474631    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:40:01.474704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:01.474679    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:40:02.075863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:02.075804    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" event={"ID":"32c9470c-c9cb-41f2-be69-23b1685f1191","Type":"ContainerStarted","Data":"37da3dcdbf8df87074444099ce7b325f7f06118ff59f3376d60bb7f7826df798"}
Oct 30 21:40:02.076077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:02.075884    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" event={"ID":"32c9470c-c9cb-41f2-be69-23b1685f1191","Type":"ContainerStarted","Data":"f98689de6c6cddb9692fa31904d7f15bd8073021fa246e5daa33ab04584293c8"}
Oct 30 21:40:02.119646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:02.119588    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podStartSLOduration=2.119572997 podStartE2EDuration="2.119572997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:02.1164378 +0000 UTC m=+2384.890837738" watchObservedRunningTime="2026-10-30 21:40:02.119572997 +0000 UTC m=+2384.893972936"
Oct 30 21:40:02.225846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:02.225799    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:03.073514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:03.073473    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:03.080316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:03.080288    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:03.082853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:03.082791    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:03.083749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:03.083731    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:40:03.831929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:03.831892    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:04.528160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.528111    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:04.699250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.699178    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:40:04.700371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.700347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:04.834232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834195    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834414 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834323    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834414 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834391    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834424    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834445    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834462    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834524    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834594    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834621    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket\") pod \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\" (UID: \"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe\") "
Oct 30 21:40:04.834814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834699    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:04.834814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834770    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:04.834992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.834968    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:04.835269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835246    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:04.835357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835340    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:04.835512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835492    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.835565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835520    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.835565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835536    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.835565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835551    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.835662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.835565    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.836691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.836672    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "kube-api-access-mb498". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:04.836907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.836884    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:40:04.837331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.837308    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:04.837398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.837313    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token" pod "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" (UID: "d5a250b6-26b3-4c9c-9c5a-71c3da498cbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:04.936588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.936551    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.936588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.936582    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.936588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.936591    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:04.936858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:04.936600    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb498\" (UniqueName: \"kubernetes.io/projected/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe-kube-api-access-mb498\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:05.098899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.098864    4549 generic.go:356] "Generic (PLEG): container finished" podID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerID="a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855" exitCode=0
Oct 30 21:40:05.099097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.098924    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"
Oct 30 21:40:05.099097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.098948    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" event={"ID":"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe","Type":"ContainerDied","Data":"a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855"}
Oct 30 21:40:05.099097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.098999    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr" event={"ID":"d5a250b6-26b3-4c9c-9c5a-71c3da498cbe","Type":"ContainerDied","Data":"4f55faea2b39261005e5acc5519652189d9236fa3d8d4d7a2954cc9d8b4a8ae1"}
Oct 30 21:40:05.099097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.099022    4549 scope.go:122] "RemoveContainer" containerID="a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855"
Oct 30 21:40:05.111099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.111077    4549 scope.go:122] "RemoveContainer" containerID="6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"
Oct 30 21:40:05.125364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.125342    4549 scope.go:122] "RemoveContainer" containerID="a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855"
Oct 30 21:40:05.125745 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:05.125728    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855\": container with ID starting with a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855 not found: ID does not exist" containerID="a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855"
Oct 30 21:40:05.125802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.125754    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855"} err="failed to get container status \"a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855\": rpc error: code = NotFound desc = could not find container \"a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855\": container with ID starting with a51274abd23578350c41dcf38f3f5b5f7ae5f8cb9f153218d0ae13175e458855 not found: ID does not exist"
Oct 30 21:40:05.125802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.125772    4549 scope.go:122] "RemoveContainer" containerID="6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"
Oct 30 21:40:05.126068 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:05.126048    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d\": container with ID starting with 6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d not found: ID does not exist" containerID="6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"
Oct 30 21:40:05.126109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.126075    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d"} err="failed to get container status \"6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d\": rpc error: code = NotFound desc = could not find container \"6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d\": container with ID starting with 6f28aaa87357eb655f56c3270eff8a5895b924e6645c5d596540e6acd20e3d9d not found: ID does not exist"
Oct 30 21:40:05.157819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.157783    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"]
Oct 30 21:40:05.161259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.161232    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr"]
Oct 30 21:40:05.346973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.346939    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" path="/var/lib/kubelet/pods/d5a250b6-26b3-4c9c-9c5a-71c3da498cbe/volumes"
Oct 30 21:40:05.609396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:05.609345    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.215977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.215933    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.335176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.335137    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"]
Oct 30 21:40:06.336237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.336213    4549 cpu_manager.go:414] "containerMap: removing container" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy"
Oct 30 21:40:06.336293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.336243    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy"
Oct 30 21:40:06.336293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.336260    4549 cpu_manager.go:414] "containerMap: removing container" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy"
Oct 30 21:40:06.336293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.336268    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy"
Oct 30 21:40:06.336436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.336423    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy"
Oct 30 21:40:06.336473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.336458    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5a250b6-26b3-4c9c-9c5a-71c3da498cbe" containerName="istio-proxy"
Oct 30 21:40:06.342260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.342234    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.356612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.356582    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"]
Oct 30 21:40:06.379372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.379332    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.423149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.423117    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-55c5e0-dockercfg-n7nwg\""
Oct 30 21:40:06.428000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.427683    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-55c5e0\""
Oct 30 21:40:06.471893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.471768    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxmdm\" (UniqueName: \"kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm\") pod \"payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.472292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.472259    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.573921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.573887    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxmdm\" (UniqueName: \"kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm\") pod \"payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.574127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.574012    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.574747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.574715    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.593887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.593820    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxmdm\" (UniqueName: \"kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm\") pod \"payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.659738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.659699    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:06.970209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:06.970175    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"]
Oct 30 21:40:06.972091 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:40:06.972065    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42172e4b_91d7_4bab_8aa8_c22212cdfaa7.slice/crio-9703df7bd1fdac8831e0d7785d71c502ef4a5e3563a684cb723ba07f3fdffd67 WatchSource:0}: Error finding container 9703df7bd1fdac8831e0d7785d71c502ef4a5e3563a684cb723ba07f3fdffd67: Status 404 returned error can't find the container with id 9703df7bd1fdac8831e0d7785d71c502ef4a5e3563a684cb723ba07f3fdffd67
Oct 30 21:40:07.126688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:07.126650    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" event={"ID":"42172e4b-91d7-4bab-8aa8-c22212cdfaa7","Type":"ContainerStarted","Data":"c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601"}
Oct 30 21:40:07.126961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:07.126706    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" event={"ID":"42172e4b-91d7-4bab-8aa8-c22212cdfaa7","Type":"ContainerStarted","Data":"9703df7bd1fdac8831e0d7785d71c502ef4a5e3563a684cb723ba07f3fdffd67"}
Oct 30 21:40:07.126961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:07.126741    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:07.168669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:07.168608    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" podStartSLOduration=1.16859128 podStartE2EDuration="1.16859128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:07.165865253 +0000 UTC m=+2389.940265188" watchObservedRunningTime="2026-10-30 21:40:07.16859128 +0000 UTC m=+2389.942991265"
Oct 30 21:40:07.202406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:07.202368    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:08.734330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:08.734266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:09.058439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:09.058331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:09.197223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:09.197155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:10.300508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:10.300463    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:10.429788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:10.429754    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:10.565261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:10.565217    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:12.032921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:12.032888    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:13.605876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:13.601219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:14.752950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.752914    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"]
Oct 30 21:40:14.757346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.757321    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.779682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.779650    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"]
Oct 30 21:40:14.787391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.787261    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ad73a4\"/\"openshift-service-ca.crt\""
Oct 30 21:40:14.787556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.787443    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ad73a4\"/\"kube-root-ca.crt\""
Oct 30 21:40:14.787556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.787548    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ad73a4\"/\"rate-test-model-ad73a4-kserve-self-signed-certs\""
Oct 30 21:40:14.787718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.787643    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ad73a4\"/\"default-dockercfg-b49x2\""
Oct 30 21:40:14.834346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.834312    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.834537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.834463    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.834537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.834516    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbf6j\" (UniqueName: \"kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.834636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.834611    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.834734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.834705    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.834798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.834763    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.863010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.862959    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:14.936547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936513    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.936748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936627    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.936748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936724    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.936895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936800    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbf6j\" (UniqueName: \"kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.936895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936885    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.936989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936926    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.937026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.936977    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.937227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.937206    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.937290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.937241    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.939540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.939511    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.939671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.939595    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:14.963391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:14.963361    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbf6j\" (UniqueName: \"kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j\") pod \"rate-test-model-ad73a4-kserve-886bdfdb9-w578k\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") " pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:15.072179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:15.072141    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:15.286290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:15.283525    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:15.399001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:15.398970    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"]
Oct 30 21:40:15.399401 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:40:15.399376    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc842686b_2734_4d66_9b46_253dadde14c8.slice/crio-3e4dc562eacd8bf10f13b2b64a5b23de1a478b88e8486826f02a89e9dacf5f1b WatchSource:0}: Error finding container 3e4dc562eacd8bf10f13b2b64a5b23de1a478b88e8486826f02a89e9dacf5f1b: Status 404 returned error can't find the container with id 3e4dc562eacd8bf10f13b2b64a5b23de1a478b88e8486826f02a89e9dacf5f1b
Oct 30 21:40:16.189035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:16.188995    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k" event={"ID":"c842686b-2734-4d66-9b46-253dadde14c8","Type":"ContainerStarted","Data":"07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5"}
Oct 30 21:40:16.189035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:16.189041    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k" event={"ID":"c842686b-2734-4d66-9b46-253dadde14c8","Type":"ContainerStarted","Data":"3e4dc562eacd8bf10f13b2b64a5b23de1a478b88e8486826f02a89e9dacf5f1b"}
Oct 30 21:40:16.230255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:16.230201    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k" podStartSLOduration=2.230185299 podStartE2EDuration="2.230185299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:16.227934293 +0000 UTC m=+2399.002334231" watchObservedRunningTime="2026-10-30 21:40:16.230185299 +0000 UTC m=+2399.004585238"
Oct 30 21:40:16.919044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:16.919007    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:17.075594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:17.075552    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:17.417794 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:17.417757    4549 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc842686b_2734_4d66_9b46_253dadde14c8.slice/crio-3e4dc562eacd8bf10f13b2b64a5b23de1a478b88e8486826f02a89e9dacf5f1b\": RecentStats: unable to find data in memory cache]"
Oct 30 21:40:18.160130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:18.160097    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:40:18.307642 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:18.307594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:18.547211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:18.547113    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:40:18.553002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:18.552978    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:40:19.352572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:19.352533    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:19.921968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:19.921933    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:21.330325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:21.330288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:22.708777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.708740    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"]
Oct 30 21:40:22.715225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.715200    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.733181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.733154    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-f5bef0\"/\"default-dockercfg-pkkbj\""
Oct 30 21:40:22.737158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.737129    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"]
Oct 30 21:40:22.744942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.744901    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-f5bef0\"/\"kube-root-ca.crt\""
Oct 30 21:40:22.744942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.744902    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-f5bef0\"/\"auth-test-model-f5bef0-kserve-self-signed-certs\""
Oct 30 21:40:22.745199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.744905    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-f5bef0\"/\"openshift-service-ca.crt\""
Oct 30 21:40:22.789106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.789070    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:22.859430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.859393    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q669t\" (UniqueName: \"kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.859649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.859570    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.859649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.859608    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.859649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.859636    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.859780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.859754    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.860002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.859985    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.925009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.924973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:22.961767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.961665    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.961883    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.961956    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q669t\" (UniqueName: \"kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.962075    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.962105    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.962131    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.962572    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.962887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.962867    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:22.963138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:22.963117    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:23.003049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.003020    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q669t\" (UniqueName: \"kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:23.003489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.003462    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:23.014918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.009003    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm\") pod \"auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") " pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:23.030818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.029284    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:23.346355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.346326    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"]
Oct 30 21:40:23.348135 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:40:23.348104    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod072a5017_9232_45bd_b22e_ce7cda4042ff.slice/crio-fa7d421c5b06c4aa33d786754475d7607085aa4d7bf2a6cde0c6d604a1a521f5 WatchSource:0}: Error finding container fa7d421c5b06c4aa33d786754475d7607085aa4d7bf2a6cde0c6d604a1a521f5: Status 404 returned error can't find the container with id fa7d421c5b06c4aa33d786754475d7607085aa4d7bf2a6cde0c6d604a1a521f5
Oct 30 21:40:23.407157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.407126    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"]
Oct 30 21:40:23.407401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:23.407375    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" containerID="cri-o://dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec" gracePeriod=30
Oct 30 21:40:24.262584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:24.262544    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt" event={"ID":"072a5017-9232-45bd-b22e-ce7cda4042ff","Type":"ContainerStarted","Data":"84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1"}
Oct 30 21:40:24.262584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:24.262587    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt" event={"ID":"072a5017-9232-45bd-b22e-ce7cda4042ff","Type":"ContainerStarted","Data":"fa7d421c5b06c4aa33d786754475d7607085aa4d7bf2a6cde0c6d604a1a521f5"}
Oct 30 21:40:24.304516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:24.304456    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt" podStartSLOduration=2.304441384 podStartE2EDuration="2.304441384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:24.301041533 +0000 UTC m=+2407.075441472" watchObservedRunningTime="2026-10-30 21:40:24.304441384 +0000 UTC m=+2407.078841323"
Oct 30 21:40:24.327344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:24.327303    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:25.030119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:25.030079    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:25.075454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:25.075420    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:25.088944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:25.088913    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:25.281477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:25.281393    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:40:25.803758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:25.803717    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:26.257869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:26.257816    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:40:26.258051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:26.257893    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:40:26.576004 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:26.575959    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:40:27.153232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:27.153200    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:28.540956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.540924    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:28.716405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.716380    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:40:28.843964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.843932    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.843984    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844059    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844107    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844151    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844186    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844281    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844445    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844445    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:28.844610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844517    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:28.844610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844539    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket\") pod \"af385494-acff-455b-b1fe-0f561f909fc9\" (UID: \"af385494-acff-455b-b1fe-0f561f909fc9\") "
Oct 30 21:40:28.844716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844657    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:28.844770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844722    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:28.845004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.844976    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:28.845428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.845398    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.845539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.845428    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af385494-acff-455b-b1fe-0f561f909fc9-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.845539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.845444    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.845539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.845457    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.845539 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.845470    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.846641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.846616    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:40:28.846791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.846616    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:40:28.847100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.847082    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "kube-api-access-5pwv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:28.847179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.847098    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token" pod "af385494-acff-455b-b1fe-0f561f909fc9" (UID: "af385494-acff-455b-b1fe-0f561f909fc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:28.947271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.947188    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.947271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.947219    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af385494-acff-455b-b1fe-0f561f909fc9-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.947271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.947228    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af385494-acff-455b-b1fe-0f561f909fc9-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:28.947271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:28.947237    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pwv5\" (UniqueName: \"kubernetes.io/projected/af385494-acff-455b-b1fe-0f561f909fc9-kube-api-access-5pwv5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:40:29.299029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.298932    4549 generic.go:356] "Generic (PLEG): container finished" podID="af385494-acff-455b-b1fe-0f561f909fc9" containerID="dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec" exitCode=0
Oct 30 21:40:29.299029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.298988    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"
Oct 30 21:40:29.299029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.299016    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" event={"ID":"af385494-acff-455b-b1fe-0f561f909fc9","Type":"ContainerDied","Data":"dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec"}
Oct 30 21:40:29.299313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.299070    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb" event={"ID":"af385494-acff-455b-b1fe-0f561f909fc9","Type":"ContainerDied","Data":"4a8aa02c9c2f99c9cf6325b4e4b2dd0a7bc4eba346250e7021f1a49f50dedc09"}
Oct 30 21:40:29.299313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.299091    4549 scope.go:122] "RemoveContainer" containerID="dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec"
Oct 30 21:40:29.310108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.310086    4549 scope.go:122] "RemoveContainer" containerID="dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec"
Oct 30 21:40:29.310397 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:29.310377    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec\": container with ID starting with dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec not found: ID does not exist" containerID="dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec"
Oct 30 21:40:29.310476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.310404    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec"} err="failed to get container status \"dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec\": rpc error: code = NotFound desc = could not find container \"dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec\": container with ID starting with dcd640ecb4a7d6aa239d967d046196eb42c683d0e66ec2a4585c40b6c92defec not found: ID does not exist"
Oct 30 21:40:29.356666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.356631    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"]
Oct 30 21:40:29.362517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.362484    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb"]
Oct 30 21:40:29.934386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:29.934347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:31.345199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:31.345164    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af385494-acff-455b-b1fe-0f561f909fc9" path="/var/lib/kubelet/pods/af385494-acff-455b-b1fe-0f561f909fc9/volumes"
Oct 30 21:40:31.506700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:31.506661    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:32.926718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:32.926682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:33.030383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:33.030341    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:33.043964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:33.043933    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:33.346951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:33.346923    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:40:33.442272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:33.442239    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:34.304066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.304031    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"]
Oct 30 21:40:34.304760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.304744    4549 cpu_manager.go:414] "containerMap: removing container" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy"
Oct 30 21:40:34.304808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.304762    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy"
Oct 30 21:40:34.304899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.304888    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="af385494-acff-455b-b1fe-0f561f909fc9" containerName="istio-proxy"
Oct 30 21:40:34.309996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.309970    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.331509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.331457    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"]
Oct 30 21:40:34.332101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.332079    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-55c5e0-serving-cert\""
Oct 30 21:40:34.406484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.406446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:34.427123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.427082    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.427370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.427345    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcdnv\" (UniqueName: \"kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.427671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.427654    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.427882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.427824    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.525423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.525390    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:34.529054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.529023    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.529221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.529104    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.529315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.529257    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.529388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.529375    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcdnv\" (UniqueName: \"kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.529914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.529890    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.529914 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:34.529909    4549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-serving-cert: secret "maas-api-e2e-shared-a-w5-55c5e0-serving-cert" not found
Oct 30 21:40:34.530087 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:34.529993    4549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls podName:59724577-6c37-4b72-a404-725cb32ff7b3 nodeName:}" failed. No retries permitted until 2026-10-30 21:40:35.029974643 +0000 UTC m=+2417.804374567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls") pod "maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" (UID: "59724577-6c37-4b72-a404-725cb32ff7b3") : secret "maas-api-e2e-shared-a-w5-55c5e0-serving-cert" not found
Oct 30 21:40:34.532820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.532772    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.551702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.551673    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcdnv\" (UniqueName: \"kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:34.678964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:34.678924    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:35.036665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:35.036571    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:35.039327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:35.039299    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:35.224645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:35.224604    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:35.545163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:35.545132    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"]
Oct 30 21:40:35.546663 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:40:35.546633    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59724577_6c37_4b72_a404_725cb32ff7b3.slice/crio-becc99bf33ca6b56bdf59551acdd1a179f409bf999de8fe5fb632589da7352ab WatchSource:0}: Error finding container becc99bf33ca6b56bdf59551acdd1a179f409bf999de8fe5fb632589da7352ab: Status 404 returned error can't find the container with id becc99bf33ca6b56bdf59551acdd1a179f409bf999de8fe5fb632589da7352ab
Oct 30 21:40:36.031409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:36.031373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:36.465139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:36.465091    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" event={"ID":"59724577-6c37-4b72-a404-725cb32ff7b3","Type":"ContainerStarted","Data":"bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5"}
Oct 30 21:40:36.465326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:36.465153    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" event={"ID":"59724577-6c37-4b72-a404-725cb32ff7b3","Type":"ContainerStarted","Data":"becc99bf33ca6b56bdf59551acdd1a179f409bf999de8fe5fb632589da7352ab"}
Oct 30 21:40:36.465326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:36.465181    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:36.505575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:36.505510    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" podStartSLOduration=2.150323535 podStartE2EDuration="2.505495638s" totalImagesPullingTime="355.172103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:36.503148496 +0000 UTC m=+2419.277548437" watchObservedRunningTime="2026-10-30 21:40:36.505495638 +0000 UTC m=+2419.279895571"
Oct 30 21:40:37.517567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:37.517451    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:38.911187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:38.911132    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:40.281945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:40.281895    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:41.805382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:41.805341    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:42.477709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:42.477677    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:40:43.311977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:43.311922    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:44.961328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:44.961288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.179973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:46.179863    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.430213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:46.430128    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.136201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:48.136114    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.305333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:48.305297    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.434230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:48.434147    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:49.986822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:49.986780    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:51.614002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:51.613960    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:53.079961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:53.079914    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:54.526559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:54.526506    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:56.082053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:56.082019    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:56.578234 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:40:56.578192    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:40:57.059219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.059123    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"]
Oct 30 21:40:57.063893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.063817    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.080991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.080958    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"]
Oct 30 21:40:57.089365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.089180    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"default-dockercfg-wz7zg\""
Oct 30 21:40:57.090724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.090696    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"openshift-service-ca.crt\""
Oct 30 21:40:57.090970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.090788    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"kube-root-ca.crt\""
Oct 30 21:40:57.090970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.090894    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-70d3a9\"/\"rate-test-model-70d3a9-kserve-self-signed-certs\""
Oct 30 21:40:57.111544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.111503    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.111763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.111577    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.111763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.111623    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf9wg\" (UniqueName: \"kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.111763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.111648    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.111763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.111731    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.112091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.111773    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.141716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.141671    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:57.213602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.213566    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.213820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.213644    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.213820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.213698    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf9wg\" (UniqueName: \"kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.213820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.213733    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.213820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.213868    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.214162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.213985    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.214162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.214122    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.214289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.214170    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.214417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.214395    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.216459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.216436    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.216865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.216844    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.233815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.233773    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf9wg\" (UniqueName: \"kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg\") pod \"rate-test-model-70d3a9-kserve-85b885599c-pk59l\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") " pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.388268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.387102    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:57.524564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.524532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:57.640464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.640376    4549 generic.go:356] "Generic (PLEG): container finished" podID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerID="e6981a78c4ffd977f09c2c5f95ba7e73bbe9767e37e9067f56f4ff6ee33bbd30" exitCode=0
Oct 30 21:40:57.640464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.640451    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" event={"ID":"2c5bc90d-b046-4a1f-88de-93bbb58aac27","Type":"ContainerDied","Data":"e6981a78c4ffd977f09c2c5f95ba7e73bbe9767e37e9067f56f4ff6ee33bbd30"}
Oct 30 21:40:57.640953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.640936    4549 scope.go:122] "RemoveContainer" containerID="e6981a78c4ffd977f09c2c5f95ba7e73bbe9767e37e9067f56f4ff6ee33bbd30"
Oct 30 21:40:57.642678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.642644    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:40:57.642768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.642717    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:40:57.642768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.642760    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:40:57.680625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.680595    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:40:57.704107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:57.702022    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"]
Oct 30 21:40:57.711927 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:40:57.711899    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc1797d3_96e0_410e_b523_f6a1975ca35a.slice/crio-3088b6295db25486a7af3b6d0ff769819c2a71955058eba4f4117650f12f7f2a WatchSource:0}: Error finding container 3088b6295db25486a7af3b6d0ff769819c2a71955058eba4f4117650f12f7f2a: Status 404 returned error can't find the container with id 3088b6295db25486a7af3b6d0ff769819c2a71955058eba4f4117650f12f7f2a
Oct 30 21:40:58.652281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:58.652242    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" event={"ID":"2c5bc90d-b046-4a1f-88de-93bbb58aac27","Type":"ContainerStarted","Data":"aeb7fec8f8853dde6cb299414698f7e3a10daf323f3dfda66256bdad6bfccd10"}
Oct 30 21:40:58.653995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:58.653967    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l" event={"ID":"fc1797d3-96e0-410e-b523-f6a1975ca35a","Type":"ContainerStarted","Data":"744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947"}
Oct 30 21:40:58.654164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:58.654002    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l" event={"ID":"fc1797d3-96e0-410e-b523-f6a1975ca35a","Type":"ContainerStarted","Data":"3088b6295db25486a7af3b6d0ff769819c2a71955058eba4f4117650f12f7f2a"}
Oct 30 21:40:58.865557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:58.865525    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:40:58.866691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:58.866665    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 21:40:58.866823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:58.866717    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 21:40:59.211676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:59.211638    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:59.388089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:59.388057    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:40:59.866215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:59.866170    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 21:40:59.866655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:40:59.866234    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 21:41:00.849151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:00.849117    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:00.866220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:00.866185    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 21:41:00.866621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:00.866254    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 21:41:01.866199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:01.866162    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 21:41:01.866400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:01.866217    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 21:41:02.392857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:02.391815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:02.865796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:02.865758    4549 patch_prober.go:29] interesting pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 21:41:02.866023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:02.865814    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 21:41:03.922068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:03.922029    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:41:03.922946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:03.922916    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:41:03.925124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:03.925096    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:41:03.933609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:03.933546    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:03.981093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:03.981034    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l" podStartSLOduration=6.981015932 podStartE2EDuration="6.981015932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:58.789345136 +0000 UTC m=+2441.563745075" watchObservedRunningTime="2026-10-30 21:41:03.981015932 +0000 UTC m=+2446.755415870"
Oct 30 21:41:05.291119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:05.291073    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:06.737149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:06.737117    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:07.387598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:07.387561    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:41:07.401022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:07.400988    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:41:07.750000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:07.749917    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:41:08.316976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:08.316939    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:09.267734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:09.267691    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:09.839098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:09.839052    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:10.914124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:10.914084    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:11.347536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:11.347501    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:11.581434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:11.581402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:11.926152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:11.926067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:12.089398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:12.089360    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:12.783983 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:12.783951    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:14.183962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:14.183917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:15.836475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:15.836441    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:15.980778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:15.980742    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:16.113573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:16.113489    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:17.493129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:17.493085    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:17.637307 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:17.637274    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:17.776962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:17.776878    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:18.949626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:18.949591    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:19.133819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:19.133789    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:20.832706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:20.832666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:22.439238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:22.439202    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:24.161710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:24.161666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:25.754412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:25.754379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:26.152733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:26.152689    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:26.600223 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:41:26.600181    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:41:27.335331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:27.335293    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:28.749973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:28.749935    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:30.196480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:30.196441    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:31.726597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.726567    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:31.936822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.936781    4549 generic.go:356] "Generic (PLEG): container finished" podID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerID="37da3dcdbf8df87074444099ce7b325f7f06118ff59f3376d60bb7f7826df798" exitCode=0
Oct 30 21:41:31.937022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.936855    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" event={"ID":"32c9470c-c9cb-41f2-be69-23b1685f1191","Type":"ContainerDied","Data":"37da3dcdbf8df87074444099ce7b325f7f06118ff59f3376d60bb7f7826df798"}
Oct 30 21:41:31.937366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.937346    4549 scope.go:122] "RemoveContainer" containerID="37da3dcdbf8df87074444099ce7b325f7f06118ff59f3376d60bb7f7826df798"
Oct 30 21:41:31.939164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.939134    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:41:31.939241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.939205    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:41:31.939241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:31.939237    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:41:32.946112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:32.946073    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" event={"ID":"32c9470c-c9cb-41f2-be69-23b1685f1191","Type":"ContainerStarted","Data":"c5df2e9b9bde63f28ea45fc6974a67f4f542529ff1932b54d36f6d2e68759aa5"}
Oct 30 21:41:33.088304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:33.088268    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:41:33.089047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:33.089025    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:41:33.096634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:33.096594    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 21:41:33.096949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:33.096913    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 21:41:33.415065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:33.415030    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:34.086346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:34.086301    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 21:41:34.088645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:34.088610    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 21:41:35.079856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:35.078903    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 21:41:35.079856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:35.078999    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 21:41:35.081361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:35.081314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:36.084858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:36.077324    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 21:41:36.084858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:36.077392    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 21:41:36.745282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:36.745231    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:37.080315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:37.080273    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 21:41:37.080639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:37.080606    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 21:41:38.081289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:38.081246    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 21:41:38.082679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:38.082645    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 21:41:38.316272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:38.316224    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:39.078597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:39.078564    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:41:39.079739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:39.079716    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:41:39.969109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:39.969074    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:40.552074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:40.552035    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:41.478928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:41.478822    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:42.907670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:42.907642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:44.409503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:44.409464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:45.824140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:45.824063    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:47.287852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:47.285395    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:48.591077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.591041    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"]
Oct 30 21:41:48.595639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.595613    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.612950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.612915    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-da52fd-serving-cert\""
Oct 30 21:41:48.630601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.630555    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"]
Oct 30 21:41:48.677231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.677194    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.677481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.677366    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.677481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.677405    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.677481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.677463    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkwqc\" (UniqueName: \"kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.692250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.692220    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:48.779327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.779292    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.779327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.779331    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.779593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.779473    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkwqc\" (UniqueName: \"kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.779716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.779699    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.780098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.780077    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.782249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.782221    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.782390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.782312    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.799565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.799533    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkwqc\" (UniqueName: \"kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc\") pod \"maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:48.837862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.836405    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:48.911584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:48.911495    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:49.221690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:49.221658    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"]
Oct 30 21:41:49.224287 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:41:49.224260    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c7073f_2822_494d_8610_fc747fc2affe.slice/crio-8136e30c24e57bfe974fd08abb569e89334c95e3d700cda3ba7d55f44e9de623 WatchSource:0}: Error finding container 8136e30c24e57bfe974fd08abb569e89334c95e3d700cda3ba7d55f44e9de623: Status 404 returned error can't find the container with id 8136e30c24e57bfe974fd08abb569e89334c95e3d700cda3ba7d55f44e9de623
Oct 30 21:41:50.087428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:50.087387    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" event={"ID":"59c7073f-2822-494d-8610-fc747fc2affe","Type":"ContainerStarted","Data":"1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e"}
Oct 30 21:41:50.087973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:50.087455    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:50.087973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:50.087474    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" event={"ID":"59c7073f-2822-494d-8610-fc747fc2affe","Type":"ContainerStarted","Data":"8136e30c24e57bfe974fd08abb569e89334c95e3d700cda3ba7d55f44e9de623"}
Oct 30 21:41:50.164738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:50.164679    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" podStartSLOduration=1.735107067 podStartE2EDuration="2.164660092s" totalImagesPullingTime="429.553025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:50.161869646 +0000 UTC m=+2492.936269577" watchObservedRunningTime="2026-10-30 21:41:50.164660092 +0000 UTC m=+2492.939060031"
Oct 30 21:41:50.400234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:50.400150    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:51.920530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:51.920486    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:53.411280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:53.411240    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:54.848267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:54.848217    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:56.101104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:56.101072    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:41:56.304020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:56.303961    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:56.574805 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:41:56.574767    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:41:57.705295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:57.705240    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:59.114024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:41:59.113964    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:00.513228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:00.513182    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:01.917078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:01.917007    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:03.341906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:03.341870    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:04.809686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:04.809651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:06.238772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:06.238733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:07.718081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:07.718047    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:09.111589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:09.111550    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:09.420578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:09.420479    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:10.582130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:10.582100    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:12.110799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:12.110749    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:13.995247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:13.995191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:14.205104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:14.205067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:15.567533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:15.567477    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:17.101737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:17.101704    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:18.660181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:18.660146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.138442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:20.138406    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:21.644282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:21.644244    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:23.102131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:23.102091    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:24.679940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:24.679898    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:26.089852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:26.089794    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:26.576360 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:42:26.576319    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:42:27.545293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:27.545257    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:27.688004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:27.687958    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.144248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:29.144208    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:30.626313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:30.626270    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:32.026309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:32.026268    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:33.413721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:33.413675    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:34.819204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:34.819165    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:36.361010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:36.360969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:37.823140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:37.823103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:38.268865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:38.268752    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:39.155718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.155667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:39.292361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.292328    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:39.605480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.605437    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"]
Oct 30 21:42:39.611900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.611874    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.635874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.635842    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-cdcf9d-openshift-default-dockercfg-892tx\""
Oct 30 21:42:39.653567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.653532    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"]
Oct 30 21:42:39.715792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.715744    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.716094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.715881    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.716094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.716062    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.716094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.716089    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.716279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.716134    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.716279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.716165    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.749659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.749607    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.751053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.751025    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rstsg\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.757527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.757373    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.762358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.762296    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:39.877066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.876977    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877014    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877047    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877066    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877175    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877212    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rstsg\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877246    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877319    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877411    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877494    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877552    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.877902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.877880    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.878060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.878029    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.878198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.878165    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.884402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.884374    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.895450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.894575    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.897318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.897283    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rstsg\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.897735 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.897694    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token\") pod \"e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") " pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:39.930021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:39.929988    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:40.057312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.057179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:40.259971 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:42:40.259932    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83daaa67_1780_4025_8921_3466c3867f59.slice/crio-e76767d3bf2daac0a0de854eeab5c53b97746a8e061c940409ec1a3993abba6b WatchSource:0}: Error finding container e76767d3bf2daac0a0de854eeab5c53b97746a8e061c940409ec1a3993abba6b: Status 404 returned error can't find the container with id e76767d3bf2daac0a0de854eeab5c53b97746a8e061c940409ec1a3993abba6b
Oct 30 21:42:40.260590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.260568    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"]
Oct 30 21:42:40.262934 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.262900    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:42:40.263025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.262969    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:42:40.263025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.263001    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:42:40.478852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.478521    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" event={"ID":"83daaa67-1780-4025-8921-3466c3867f59","Type":"ContainerStarted","Data":"a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"}
Oct 30 21:42:40.478852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.478582    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" event={"ID":"83daaa67-1780-4025-8921-3466c3867f59","Type":"ContainerStarted","Data":"e76767d3bf2daac0a0de854eeab5c53b97746a8e061c940409ec1a3993abba6b"}
Oct 30 21:42:40.521805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.521675    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podStartSLOduration=1.521656801 podStartE2EDuration="1.521656801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:40.520327757 +0000 UTC m=+2543.294727696" watchObservedRunningTime="2026-10-30 21:42:40.521656801 +0000 UTC m=+2543.296056741"
Oct 30 21:42:40.929148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.929114    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:40.930972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:40.930952    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:41.937988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:41.937949    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:42.502108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:42.502074    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:42.502972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:42.502953    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:42:42.604470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:42.604416    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:42.855774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:42.855738    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:43.008176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:43.008141    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.379259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:44.379222    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.535606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:44.535568    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.730577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:44.730437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:46.039346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:46.039308    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:47.437542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:47.437508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:48.833537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:48.833501    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:50.423813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:50.423772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:51.792140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:51.792100    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.212350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:53.212314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.336352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:53.336306    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:54.785566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:54.785532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:56.202927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:56.202888    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:56.575873 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:42:56.575804    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:42:57.664003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:57.663954    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:59.117885 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:42:59.117783    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:00.617047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:00.617015    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:02.097145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:02.097102    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:03.504289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:03.504250    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:04.960903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:04.960861    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:06.649137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:06.649098    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.919145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:07.919103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:09.320105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:09.320068    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:09.564518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:09.564479    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.947066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:10.947030    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:12.421437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:12.421360    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:13.907879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:13.907806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:15.413523 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:15.413488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:15.542296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:15.542267    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:15.672435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:15.672342    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:17.021378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:17.021344    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:17.232183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:17.232148    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:18.534334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:18.534302    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:20.315362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:20.315323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:21.893477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:21.893437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:23.365362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:23.365331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:24.979080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:24.979037    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:26.561250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:26.561212    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:26.593073 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:43:26.593032    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:43:28.176130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:28.176095    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:29.747686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:29.747627    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:31.256113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:31.256074    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:31.782573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:31.782527    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:31.924131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:31.924093    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:32.743614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:32.743570    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:34.374263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:34.374180    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:35.985289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:35.985246    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:37.561614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:37.561570    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:39.435668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:39.435624    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:41.089862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:41.088020    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:42.716683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:42.716648    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:44.384135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:44.384079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:45.992941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:45.992904    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:47.782776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:47.782743    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:49.478855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:49.477292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:51.322746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:51.322704    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:53.205652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:53.205607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:54.880385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:54.880345    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:56.575212 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:43:56.575177    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:43:56.725861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:56.725802    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:58.256970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:58.256915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:59.673416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:59.673379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:59.822682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:43:59.822644    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:00.051410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:00.051323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:01.529604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:01.529554    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:03.072134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:03.072101    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:04.636867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:04.630382    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:06.153122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:06.153090    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:07.744248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:07.744209    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:09.343209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:09.343119    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:09.738366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:09.738286    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:11.015863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:11.015806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:12.837930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:12.837882    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.700284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:14.700247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:16.455179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:16.453797    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:18.127210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:18.127171    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:20.017328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:20.017266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:21.713902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:21.713768    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:23.233381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:23.233344    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.357270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.357235    4549 generic.go:356] "Generic (PLEG): container finished" podID="83daaa67-1780-4025-8921-3466c3867f59" containerID="a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e" exitCode=0
Oct 30 21:44:24.357741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.357298    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" event={"ID":"83daaa67-1780-4025-8921-3466c3867f59","Type":"ContainerDied","Data":"a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"}
Oct 30 21:44:24.357788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.357768    4549 scope.go:122] "RemoveContainer" containerID="a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"
Oct 30 21:44:24.359609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.359578    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:44:24.359710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.359654    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:44:24.359710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.359687    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:44:24.814222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:24.814132    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:25.365411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:25.365373    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" event={"ID":"83daaa67-1780-4025-8921-3466c3867f59","Type":"ContainerStarted","Data":"de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7"}
Oct 30 21:44:25.930751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:25.930710    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:44:25.932464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:25.932437    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 21:44:25.932591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:25.932493    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 21:44:26.589544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:26.589500    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:26.633458 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:44:26.633383    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:44:26.931368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:26.931285    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 21:44:26.931368 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:26.931344    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 21:44:27.502884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:27.502853    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:44:27.930518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:27.930477    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 21:44:27.930998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:27.930562    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 21:44:28.265668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:28.265582    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:28.930996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:28.930955    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 21:44:28.931425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:28.931016    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 21:44:29.930532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:29.930496    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 21:44:29.930745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:29.930551    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 21:44:30.082260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:30.082218    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:30.935975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:30.935943    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:44:30.937165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:30.937141    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:44:32.022164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:32.022055    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:33.216371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:33.216330    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:33.361611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:33.361573    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:34.302452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:34.302412    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:35.216927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:35.216894    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:36.000378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:36.000312    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:37.305859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:37.304846    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"]
Oct 30 21:44:37.305859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:37.305133    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" containerID="cri-o://2e3b2433d042239dbfcf779e364e078152f9b81f8a05ea87ce1bf311dbe3b21b" gracePeriod=30
Oct 30 21:44:37.865012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:37.864964    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:39.236904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:39.236868    4549 patch_prober.go:29] interesting pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:44:39.237296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:39.236942    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:44:39.311811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:39.311773    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:39.493392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:39.493287    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:40.573348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:40.573312    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"]
Oct 30 21:44:40.742532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:40.742491    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:41.202380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:41.202341    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:42.559548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.559519    4549 generic.go:356] "Generic (PLEG): container finished" podID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerID="2e3b2433d042239dbfcf779e364e078152f9b81f8a05ea87ce1bf311dbe3b21b" exitCode=0
Oct 30 21:44:42.559899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.559601    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" event={"ID":"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399","Type":"ContainerDied","Data":"2e3b2433d042239dbfcf779e364e078152f9b81f8a05ea87ce1bf311dbe3b21b"}
Oct 30 21:44:42.589291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.589231    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:44:42.726280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726244    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726283    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726337    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726378    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726406    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726452    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726481    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726685    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726745    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket\") pod \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\" (UID: \"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399\") "
Oct 30 21:44:42.726869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726749    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:42.726869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726758    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:42.727105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726880    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:42.727105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.726942    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:42.727348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.727273    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:42.727608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.727586    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.727684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.727609    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.727684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.727623    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.727684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.727636    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.727684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.727652    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.729062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.729033    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:44:42.729162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.729073    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:42.729162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.729155    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "kube-api-access-ztdjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:42.729429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.729409    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token" pod "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" (UID: "15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:42.829119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.829079    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.829119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.829110    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.829119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.829119    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.829451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.829132    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztdjf\" (UniqueName: \"kubernetes.io/projected/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399-kube-api-access-ztdjf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:42.859061 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:42.858815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:43.568899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:43.568821    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v" event={"ID":"15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399","Type":"ContainerDied","Data":"1cf3611a8b3c93a49bd1925306139f7a7708438f8333d19246e965a2045ec5be"}
Oct 30 21:44:43.568899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:43.568895    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"
Oct 30 21:44:43.569426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:43.568940    4549 scope.go:122] "RemoveContainer" containerID="2e3b2433d042239dbfcf779e364e078152f9b81f8a05ea87ce1bf311dbe3b21b"
Oct 30 21:44:43.637687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:43.637604    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"]
Oct 30 21:44:43.643565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:43.643538    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v"]
Oct 30 21:44:44.472616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:44.472581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:45.347935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:45.347896    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" path="/var/lib/kubelet/pods/15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399/volumes"
Oct 30 21:44:46.058437 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:46.058400    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:48.228449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:48.228406    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:49.982855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:49.982059    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:51.726807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:51.726770    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:52.424881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:52.424836    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.179304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:53.179247    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.628810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:53.628771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.866153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:53.866115    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:55.600076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.600036    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs" podUID="eece88f7-f624-4554-bcea-c1091bc0390f" containerName="main" containerID="cri-o://8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac" gracePeriod=45
Oct 30 21:44:55.725389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.725352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:55.900172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.900139    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:44:55.965669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.965633    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n\" (UniqueName: \"kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n\") pod \"eece88f7-f624-4554-bcea-c1091bc0390f\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") "
Oct 30 21:44:55.965895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.965714    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache\") pod \"eece88f7-f624-4554-bcea-c1091bc0390f\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") "
Oct 30 21:44:55.965895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.965765    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir\") pod \"eece88f7-f624-4554-bcea-c1091bc0390f\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") "
Oct 30 21:44:55.965895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.965808    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm\") pod \"eece88f7-f624-4554-bcea-c1091bc0390f\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") "
Oct 30 21:44:55.965895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.965879    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home\") pod \"eece88f7-f624-4554-bcea-c1091bc0390f\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") "
Oct 30 21:44:55.966212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.965924    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs\" (UniqueName: \"kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs\") pod \"eece88f7-f624-4554-bcea-c1091bc0390f\" (UID: \"eece88f7-f624-4554-bcea-c1091bc0390f\") "
Oct 30 21:44:55.966301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.966252    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir" pod "eece88f7-f624-4554-bcea-c1091bc0390f" (UID: "eece88f7-f624-4554-bcea-c1091bc0390f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:55.966694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.966666    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:55.966973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.966949    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache" pod "eece88f7-f624-4554-bcea-c1091bc0390f" (UID: "eece88f7-f624-4554-bcea-c1091bc0390f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:55.969001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.968972    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs" pod "eece88f7-f624-4554-bcea-c1091bc0390f" (UID: "eece88f7-f624-4554-bcea-c1091bc0390f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:55.969108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.969009    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n" pod "eece88f7-f624-4554-bcea-c1091bc0390f" (UID: "eece88f7-f624-4554-bcea-c1091bc0390f"). InnerVolumeSpecName "kube-api-access-8nz5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:55.969179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.969162    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home" pod "eece88f7-f624-4554-bcea-c1091bc0390f" (UID: "eece88f7-f624-4554-bcea-c1091bc0390f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:55.969868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:55.969796    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm" pod "eece88f7-f624-4554-bcea-c1091bc0390f" (UID: "eece88f7-f624-4554-bcea-c1091bc0390f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:56.068075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.068036    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nz5n\" (UniqueName: \"kubernetes.io/projected/eece88f7-f624-4554-bcea-c1091bc0390f-kube-api-access-8nz5n\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:56.068075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.068072    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:56.068075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.068081    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:56.068345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.068089    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eece88f7-f624-4554-bcea-c1091bc0390f-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:56.068345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.068096    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eece88f7-f624-4554-bcea-c1091bc0390f-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:44:56.367734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.367696    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:56.574715 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:44:56.574679    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:44:56.623374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.623288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:56.698320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.698285    4549 generic.go:356] "Generic (PLEG): container finished" podID="eece88f7-f624-4554-bcea-c1091bc0390f" containerID="8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac" exitCode=0
Oct 30 21:44:56.698534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.698335    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"
Oct 30 21:44:56.698534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.698339    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs" event={"ID":"eece88f7-f624-4554-bcea-c1091bc0390f","Type":"ContainerDied","Data":"8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac"}
Oct 30 21:44:56.698534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.698381    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs" event={"ID":"eece88f7-f624-4554-bcea-c1091bc0390f","Type":"ContainerDied","Data":"71ae6d3745ad8a076479e5c10a8280df3b51e447a6b8beb4fcfa9557d9740bf7"}
Oct 30 21:44:56.698534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.698404    4549 scope.go:122] "RemoveContainer" containerID="8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac"
Oct 30 21:44:56.716977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.716954    4549 scope.go:122] "RemoveContainer" containerID="8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac"
Oct 30 21:44:56.717366 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:44:56.717328    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac\": container with ID starting with 8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac not found: ID does not exist" containerID="8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac"
Oct 30 21:44:56.717481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.717364    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac"} err="failed to get container status \"8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac\": rpc error: code = NotFound desc = could not find container \"8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac\": container with ID starting with 8f7964c0bb5d46fdb71f539d3282b040f09f6d9902581474995d296d346ecbac not found: ID does not exist"
Oct 30 21:44:56.768268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.768226    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"]
Oct 30 21:44:56.784768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:56.784730    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w6-95692f/auth-test-model-95692f-kserve-76ffd8f685-fcmgs"]
Oct 30 21:44:57.347112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:57.347063    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eece88f7-f624-4554-bcea-c1091bc0390f" path="/var/lib/kubelet/pods/eece88f7-f624-4554-bcea-c1091bc0390f/volumes"
Oct 30 21:44:57.659477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:57.659388    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:58.040379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:58.040287    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:58.251672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:58.251642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:59.539386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:44:59.539347    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:00.429433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.429390    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"]
Oct 30 21:45:00.430490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.430468    4549 cpu_manager.go:414] "containerMap: removing container" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy"
Oct 30 21:45:00.430544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.430495    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy"
Oct 30 21:45:00.430544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.430511    4549 cpu_manager.go:414] "containerMap: removing container" podUID="eece88f7-f624-4554-bcea-c1091bc0390f" containerName="main"
Oct 30 21:45:00.430544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.430519    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eece88f7-f624-4554-bcea-c1091bc0390f" containerName="main"
Oct 30 21:45:00.430765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.430750    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="15bb9ec9-80f1-48e3-a32d-f3ff9c3d6399" containerName="istio-proxy"
Oct 30 21:45:00.430805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.430775    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="eece88f7-f624-4554-bcea-c1091bc0390f" containerName="main"
Oct 30 21:45:00.436395 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.436372    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.468094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.468041    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tnbrd\""
Oct 30 21:45:00.469502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.469479    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"]
Oct 30 21:45:00.480249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.480222    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"]
Oct 30 21:45:00.484610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.484590    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.499766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.499733    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c8ph\" (UniqueName: \"kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.499944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.499905    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.500054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.500034    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p657d\" (UniqueName: \"kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.500155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.500133    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.580876 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.580804    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"]
Oct 30 21:45:00.601378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.601339    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c8ph\" (UniqueName: \"kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.601589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.601461    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.601589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.601566    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p657d\" (UniqueName: \"kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.602089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.601659    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.602336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.602313    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.602620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.602461    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.657038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.656997    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p657d\" (UniqueName: \"kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.661696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.661668    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c8ph\" (UniqueName: \"kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:00.754930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.754823    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:00.801939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:00.801899    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:01.064978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.064952    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"]
Oct 30 21:45:01.066079 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:45:01.066048    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1171517a_cdb9_4140_8a44_7bacfc1e555e.slice/crio-c4bf33eb08783855ebf8db0e75d2f1cd6197e819a0f70a3fbc36f3b0f95ebecd WatchSource:0}: Error finding container c4bf33eb08783855ebf8db0e75d2f1cd6197e819a0f70a3fbc36f3b0f95ebecd: Status 404 returned error can't find the container with id c4bf33eb08783855ebf8db0e75d2f1cd6197e819a0f70a3fbc36f3b0f95ebecd
Oct 30 21:45:01.139284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.139255    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"]
Oct 30 21:45:01.140211 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:45:01.140177    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b09abf_3e57_4f38_b8d9_c3cf158208b3.slice/crio-764d40c6a17d5464d0670f475ea342df9a59395faef48f15f167e20b2669a848 WatchSource:0}: Error finding container 764d40c6a17d5464d0670f475ea342df9a59395faef48f15f167e20b2669a848: Status 404 returned error can't find the container with id 764d40c6a17d5464d0670f475ea342df9a59395faef48f15f167e20b2669a848
Oct 30 21:45:01.500227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.500186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:01.740310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.740212    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerStarted","Data":"bba175b6a6bdb7ae99dd4e8a9525986e80d2172e7f6b5e766fad6f1c4e01f884"}
Oct 30 21:45:01.740310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.740270    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerStarted","Data":"764d40c6a17d5464d0670f475ea342df9a59395faef48f15f167e20b2669a848"}
Oct 30 21:45:01.741973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.741945    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerStarted","Data":"d8f1875c34b758de5499a2c35b752a9d7c139303ef168c9069467a429db87d0c"}
Oct 30 21:45:01.742119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.741994    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerStarted","Data":"c4bf33eb08783855ebf8db0e75d2f1cd6197e819a0f70a3fbc36f3b0f95ebecd"}
Oct 30 21:45:01.790682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.790628    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" podStartSLOduration=1.790613631 podStartE2EDuration="1.790613631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:01.787205541 +0000 UTC m=+2684.561605479" watchObservedRunningTime="2026-10-30 21:45:01.790613631 +0000 UTC m=+2684.565013921"
Oct 30 21:45:01.836302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:01.836252    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" podStartSLOduration=1.836236349 podStartE2EDuration="1.836236349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:01.834182507 +0000 UTC m=+2684.608582446" watchObservedRunningTime="2026-10-30 21:45:01.836236349 +0000 UTC m=+2684.610636288"
Oct 30 21:45:02.751585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:02.751547    4549 generic.go:356] "Generic (PLEG): container finished" podID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerID="bba175b6a6bdb7ae99dd4e8a9525986e80d2172e7f6b5e766fad6f1c4e01f884" exitCode=60
Oct 30 21:45:02.752130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:02.751630    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerDied","Data":"bba175b6a6bdb7ae99dd4e8a9525986e80d2172e7f6b5e766fad6f1c4e01f884"}
Oct 30 21:45:02.752130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:02.751973    4549 scope.go:122] "RemoveContainer" containerID="bba175b6a6bdb7ae99dd4e8a9525986e80d2172e7f6b5e766fad6f1c4e01f884"
Oct 30 21:45:02.753549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:02.753517    4549 generic.go:356] "Generic (PLEG): container finished" podID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerID="d8f1875c34b758de5499a2c35b752a9d7c139303ef168c9069467a429db87d0c" exitCode=60
Oct 30 21:45:02.753648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:02.753594    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerDied","Data":"d8f1875c34b758de5499a2c35b752a9d7c139303ef168c9069467a429db87d0c"}
Oct 30 21:45:02.753869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:02.753819    4549 scope.go:122] "RemoveContainer" containerID="d8f1875c34b758de5499a2c35b752a9d7c139303ef168c9069467a429db87d0c"
Oct 30 21:45:03.267059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:03.267021    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:03.762694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:03.762656    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerStarted","Data":"a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b"}
Oct 30 21:45:03.764823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:03.764795    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerStarted","Data":"adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552"}
Oct 30 21:45:04.785619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.785577    4549 generic.go:356] "Generic (PLEG): container finished" podID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerID="a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b" exitCode=60
Oct 30 21:45:04.786175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.785669    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerDied","Data":"a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b"}
Oct 30 21:45:04.786175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.785784    4549 scope.go:122] "RemoveContainer" containerID="bba175b6a6bdb7ae99dd4e8a9525986e80d2172e7f6b5e766fad6f1c4e01f884"
Oct 30 21:45:04.786175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.786004    4549 scope.go:122] "RemoveContainer" containerID="a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b"
Oct 30 21:45:04.786396 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:04.786375    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx_odh-ai-gateway-infra(49b09abf-3e57-4f38-b8d9-c3cf158208b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3"
Oct 30 21:45:04.787916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.787896    4549 generic.go:356] "Generic (PLEG): container finished" podID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerID="adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552" exitCode=60
Oct 30 21:45:04.787997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.787969    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerDied","Data":"adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552"}
Oct 30 21:45:04.788226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.788212    4549 scope.go:122] "RemoveContainer" containerID="adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552"
Oct 30 21:45:04.788510 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:04.788492    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf_odh-ai-gateway-infra(1171517a-cdb9-4140-8a44-7bacfc1e555e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e"
Oct 30 21:45:04.798920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:04.798900    4549 scope.go:122] "RemoveContainer" containerID="d8f1875c34b758de5499a2c35b752a9d7c139303ef168c9069467a429db87d0c"
Oct 30 21:45:05.007718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:05.007684    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:05.797106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:05.797079    4549 scope.go:122] "RemoveContainer" containerID="a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b"
Oct 30 21:45:05.797605 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:05.797459    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx_odh-ai-gateway-infra(49b09abf-3e57-4f38-b8d9-c3cf158208b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3"
Oct 30 21:45:05.798720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:05.798698    4549 scope.go:122] "RemoveContainer" containerID="adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552"
Oct 30 21:45:05.799025 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:05.799004    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf_odh-ai-gateway-infra(1171517a-cdb9-4140-8a44-7bacfc1e555e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e"
Oct 30 21:45:06.846256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:06.846215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:07.423275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.423238    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"]
Oct 30 21:45:07.428996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.428976    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.448126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.448094    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"]
Oct 30 21:45:07.455692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.455660    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:45:07.506014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.505971    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lzp7\" (UniqueName: \"kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.506014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.506024    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.506249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.506220    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.607434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.607386    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lzp7\" (UniqueName: \"kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.607659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.607452    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.607659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.607601    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.608176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.608148    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.610331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.610307    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.628231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.628201    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lzp7\" (UniqueName: \"kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7\") pod \"maas-api-revoke-sub-a8303f26-nldcz\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:07.745371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:07.745283    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:08.056680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:08.056648    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"]
Oct 30 21:45:08.057347 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:45:08.057324    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2cd5025_ffd3_459b_b77f_40f7bcd3d25e.slice/crio-dd11b619fb8a2fe0968d07cb2b3d2f7bdae3ab7b118cc0341dd687ecd8edf4ef WatchSource:0}: Error finding container dd11b619fb8a2fe0968d07cb2b3d2f7bdae3ab7b118cc0341dd687ecd8edf4ef: Status 404 returned error can't find the container with id dd11b619fb8a2fe0968d07cb2b3d2f7bdae3ab7b118cc0341dd687ecd8edf4ef
Oct 30 21:45:08.409386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:08.408872    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:08.822766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:08.822714    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz" event={"ID":"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e","Type":"ContainerStarted","Data":"3a0a3ffc8979c8e2390ce0f45e846131787c4028fa4e5f8830c93a1d831c86b9"}
Oct 30 21:45:08.822966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:08.822791    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz" event={"ID":"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e","Type":"ContainerStarted","Data":"dd11b619fb8a2fe0968d07cb2b3d2f7bdae3ab7b118cc0341dd687ecd8edf4ef"}
Oct 30 21:45:08.868941 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:08.868890    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz" podStartSLOduration=1.868875786 podStartE2EDuration="1.868875786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:08.865233519 +0000 UTC m=+2691.639633470" watchObservedRunningTime="2026-10-30 21:45:08.868875786 +0000 UTC m=+2691.643275725"
Oct 30 21:45:09.818273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:09.818231    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:09.830576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:09.830542    4549 generic.go:356] "Generic (PLEG): container finished" podID="b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" containerID="3a0a3ffc8979c8e2390ce0f45e846131787c4028fa4e5f8830c93a1d831c86b9" exitCode=0
Oct 30 21:45:09.830705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:09.830625    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz" event={"ID":"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e","Type":"ContainerDied","Data":"3a0a3ffc8979c8e2390ce0f45e846131787c4028fa4e5f8830c93a1d831c86b9"}
Oct 30 21:45:10.029622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:10.029586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:11.021161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.021127    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:11.064114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.064085    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7\" (UniqueName: \"kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7\") pod \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") "
Oct 30 21:45:11.064280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.064239    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca\") pod \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") "
Oct 30 21:45:11.064405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.064388    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca\") pod \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\" (UID: \"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e\") "
Oct 30 21:45:11.064661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.064636    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca" pod "b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" (UID: "b2cd5025-ffd3-459b-b77f-40f7bcd3d25e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:11.064984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.064966    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-openshift-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:11.066430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.066401    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7" pod "b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" (UID: "b2cd5025-ffd3-459b-b77f-40f7bcd3d25e"). InnerVolumeSpecName "kube-api-access-4lzp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:11.066573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.066551    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca" pod "b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" (UID: "b2cd5025-ffd3-459b-b77f-40f7bcd3d25e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:11.166656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.166559    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-maas-api-xks-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:11.166656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.166596    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lzp7\" (UniqueName: \"kubernetes.io/projected/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e-kube-api-access-4lzp7\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:11.673809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.673764    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:11.846197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.846164    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"
Oct 30 21:45:11.846384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.846197    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz" event={"ID":"b2cd5025-ffd3-459b-b77f-40f7bcd3d25e","Type":"ContainerDied","Data":"dd11b619fb8a2fe0968d07cb2b3d2f7bdae3ab7b118cc0341dd687ecd8edf4ef"}
Oct 30 21:45:11.846384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:11.846245    4549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd11b619fb8a2fe0968d07cb2b3d2f7bdae3ab7b118cc0341dd687ecd8edf4ef"
Oct 30 21:45:13.289589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:13.289511    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:15.429878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:15.429822    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:17.207354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:17.207324    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:17.340456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:17.340427    4549 scope.go:122] "RemoveContainer" containerID="adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552"
Oct 30 21:45:17.891500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:17.891463    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerStarted","Data":"8422d2a0e268a765f213238008211c8d41e07e8c854f74bbd062d2e4df9253db"}
Oct 30 21:45:18.336992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.336961    4549 scope.go:122] "RemoveContainer" containerID="a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b"
Oct 30 21:45:18.483322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.483291    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:18.601384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.601303    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:45:18.607898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.607875    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:45:18.901042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.900928    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerStarted","Data":"e768787dd335d9df23ea5c756ccc9f71a25e0b1fab244cc95b4e91440cab586e"}
Oct 30 21:45:18.903247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.903219    4549 generic.go:356] "Generic (PLEG): container finished" podID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerID="8422d2a0e268a765f213238008211c8d41e07e8c854f74bbd062d2e4df9253db" exitCode=60
Oct 30 21:45:18.903378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.903286    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerDied","Data":"8422d2a0e268a765f213238008211c8d41e07e8c854f74bbd062d2e4df9253db"}
Oct 30 21:45:18.903378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.903340    4549 scope.go:122] "RemoveContainer" containerID="adb2e91167e63b4ff404d6091b990ff85a825224bfa909fb1e8f428d4dd13552"
Oct 30 21:45:18.903694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.903673    4549 scope.go:122] "RemoveContainer" containerID="8422d2a0e268a765f213238008211c8d41e07e8c854f74bbd062d2e4df9253db"
Oct 30 21:45:18.903994 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:18.903971    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf_odh-ai-gateway-infra(1171517a-cdb9-4140-8a44-7bacfc1e555e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e"
Oct 30 21:45:18.951923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:18.951892    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"]
Oct 30 21:45:19.035588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:19.035553    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:19.911962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:19.911924    4549 generic.go:356] "Generic (PLEG): container finished" podID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerID="e768787dd335d9df23ea5c756ccc9f71a25e0b1fab244cc95b4e91440cab586e" exitCode=60
Oct 30 21:45:19.912472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:19.912048    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerDied","Data":"e768787dd335d9df23ea5c756ccc9f71a25e0b1fab244cc95b4e91440cab586e"}
Oct 30 21:45:19.912472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:19.912136    4549 scope.go:122] "RemoveContainer" containerID="a131371721e924674f4a00fd396cff8d008a37c90ebec0abc777a9db8514f99b"
Oct 30 21:45:19.912634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:19.912604    4549 scope.go:122] "RemoveContainer" containerID="e768787dd335d9df23ea5c756ccc9f71a25e0b1fab244cc95b4e91440cab586e"
Oct 30 21:45:19.913110 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:19.913080    4549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx_odh-ai-gateway-infra(49b09abf-3e57-4f38-b8d9-c3cf158208b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3"
Oct 30 21:45:20.032599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.032560    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"]
Oct 30 21:45:20.092528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.092493    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:20.125343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.125308    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d\" (UniqueName: \"kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d\") pod \"1171517a-cdb9-4140-8a44-7bacfc1e555e\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") "
Oct 30 21:45:20.125558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.125420    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca\") pod \"1171517a-cdb9-4140-8a44-7bacfc1e555e\" (UID: \"1171517a-cdb9-4140-8a44-7bacfc1e555e\") "
Oct 30 21:45:20.125867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.125843    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca" pod "1171517a-cdb9-4140-8a44-7bacfc1e555e" (UID: "1171517a-cdb9-4140-8a44-7bacfc1e555e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:20.126089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.126075    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1171517a-cdb9-4140-8a44-7bacfc1e555e-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:20.127761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.127736    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d" pod "1171517a-cdb9-4140-8a44-7bacfc1e555e" (UID: "1171517a-cdb9-4140-8a44-7bacfc1e555e"). InnerVolumeSpecName "kube-api-access-p657d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:20.227554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.227458    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p657d\" (UniqueName: \"kubernetes.io/projected/1171517a-cdb9-4140-8a44-7bacfc1e555e-kube-api-access-p657d\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:20.914856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.909467    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:20.951288 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.951260    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"
Oct 30 21:45:20.951445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.951290    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf" event={"ID":"1171517a-cdb9-4140-8a44-7bacfc1e555e","Type":"ContainerDied","Data":"c4bf33eb08783855ebf8db0e75d2f1cd6197e819a0f70a3fbc36f3b0f95ebecd"}
Oct 30 21:45:20.951445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:20.951395    4549 scope.go:122] "RemoveContainer" containerID="8422d2a0e268a765f213238008211c8d41e07e8c854f74bbd062d2e4df9253db"
Oct 30 21:45:21.051531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.051500    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"]
Oct 30 21:45:21.059360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.059330    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-70d3a9-29889945-pz6wf"]
Oct 30 21:45:21.129033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.129003    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:21.139462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.139427    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph\" (UniqueName: \"kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph\") pod \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") "
Oct 30 21:45:21.139784 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.139757    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca\") pod \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\" (UID: \"49b09abf-3e57-4f38-b8d9-c3cf158208b3\") "
Oct 30 21:45:21.140328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.140294    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca" pod "49b09abf-3e57-4f38-b8d9-c3cf158208b3" (UID: "49b09abf-3e57-4f38-b8d9-c3cf158208b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:21.140496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.140441    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b09abf-3e57-4f38-b8d9-c3cf158208b3-maas-api-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:21.142221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.142193    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph" pod "49b09abf-3e57-4f38-b8d9-c3cf158208b3" (UID: "49b09abf-3e57-4f38-b8d9-c3cf158208b3"). InnerVolumeSpecName "kube-api-access-9c8ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:21.242167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.242084    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c8ph\" (UniqueName: \"kubernetes.io/projected/49b09abf-3e57-4f38-b8d9-c3cf158208b3-kube-api-access-9c8ph\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:45:21.345473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.345429    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" path="/var/lib/kubelet/pods/1171517a-cdb9-4140-8a44-7bacfc1e555e/volumes"
Oct 30 21:45:21.960213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.960185    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"
Oct 30 21:45:21.960684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.960217    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx" event={"ID":"49b09abf-3e57-4f38-b8d9-c3cf158208b3","Type":"ContainerDied","Data":"764d40c6a17d5464d0670f475ea342df9a59395faef48f15f167e20b2669a848"}
Oct 30 21:45:21.960684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:21.960306    4549 scope.go:122] "RemoveContainer" containerID="e768787dd335d9df23ea5c756ccc9f71a25e0b1fab244cc95b4e91440cab586e"
Oct 30 21:45:22.013412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:22.013376    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"]
Oct 30 21:45:22.025091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:22.025055    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-55c5e0-29889945-v5ztx"]
Oct 30 21:45:22.669855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:22.669794    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:23.351675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:23.351632    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" path="/var/lib/kubelet/pods/49b09abf-3e57-4f38-b8d9-c3cf158208b3/volumes"
Oct 30 21:45:23.439851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:23.439795    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:24.614743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:24.612456    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:26.173811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:26.173770    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:26.574907 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:26.574872    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:45:27.710875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:27.710842    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:29.400162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:29.400096    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:30.223046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:30.223013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:30.900189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:30.900146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:31.241304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:31.241211    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:31.453109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:31.453069    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:31.610855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:31.610706    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:33.036600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:33.036555    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:34.772520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:34.772301    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:35.347977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:35.347940    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:35.532122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:35.532081    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:36.377767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:36.377729    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:36.578100 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:36.578063    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:36.760500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:36.760404    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:37.493000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.492965    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:37.553132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553098    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"]
Oct 30 21:45:37.553862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553845    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" containerName="revoke-keys"
Oct 30 21:45:37.553862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553862    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" containerName="revoke-keys"
Oct 30 21:45:37.553960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553880    4549 cpu_manager.go:414] "containerMap: removing container" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.553960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553885    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.553960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553909    4549 cpu_manager.go:414] "containerMap: removing container" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.553960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553915    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.553960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553925    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.553960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553930    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553982    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553987    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553995    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.553999    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.554113    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.554127    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.554134    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.554141    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" containerName="revoke-keys"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.554147    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.554168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.554155    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.559283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.559260    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.580055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.580020    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"]
Oct 30 21:45:37.587320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.587293    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-203209-dockercfg-qj5h7\""
Oct 30 21:45:37.587495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.587328    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-203209\""
Oct 30 21:45:37.622355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.622312    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"]
Oct 30 21:45:37.623565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.623540    4549 cpu_manager.go:414] "containerMap: removing container" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.623645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.623571    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49b09abf-3e57-4f38-b8d9-c3cf158208b3" containerName="cleanup"
Oct 30 21:45:37.623810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.623795    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1171517a-cdb9-4140-8a44-7bacfc1e555e" containerName="cleanup"
Oct 30 21:45:37.627731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.627713    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.653228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.653193    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"]
Oct 30 21:45:37.715857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.702435    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:37.813557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.812795    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhnn2\" (UniqueName: \"kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2\") pod \"payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.813557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.813067    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.813557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.813110    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc9kw\" (UniqueName: \"kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw\") pod \"payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.813557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.813218    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.916443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.914210    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.916443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.914316    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhnn2\" (UniqueName: \"kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2\") pod \"payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.916443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.915793    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.916443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.916060    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.916443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.916110    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc9kw\" (UniqueName: \"kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw\") pod \"payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.917287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.917259    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.936006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.935971    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhnn2\" (UniqueName: \"kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2\") pod \"payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:37.937860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.937820    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc9kw\" (UniqueName: \"kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw\") pod \"payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:37.943883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:37.943859    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:38.174303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:38.174270    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:38.265899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:38.265868    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"]
Oct 30 21:45:38.268691 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:45:38.268661    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65130ab3_f5ca_44b5_ac53_d7868b783422.slice/crio-5df9f530e9b83319f28cd8d3f7f52324afde2805ea8237218c03bed6c409973a WatchSource:0}: Error finding container 5df9f530e9b83319f28cd8d3f7f52324afde2805ea8237218c03bed6c409973a: Status 404 returned error can't find the container with id 5df9f530e9b83319f28cd8d3f7f52324afde2805ea8237218c03bed6c409973a
Oct 30 21:45:38.411912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:38.411307    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:38.486302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:38.483988    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"]
Oct 30 21:45:39.083177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.083127    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn" event={"ID":"65130ab3-f5ca-44b5-ac53-d7868b783422","Type":"ContainerStarted","Data":"b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f"}
Oct 30 21:45:39.083758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.083196    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn" event={"ID":"65130ab3-f5ca-44b5-ac53-d7868b783422","Type":"ContainerStarted","Data":"5df9f530e9b83319f28cd8d3f7f52324afde2805ea8237218c03bed6c409973a"}
Oct 30 21:45:39.083758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.083225    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:39.085044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.085017    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f" event={"ID":"b832b36b-108e-4239-8c86-da86374dbba2","Type":"ContainerStarted","Data":"141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868"}
Oct 30 21:45:39.085170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.085065    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f" event={"ID":"b832b36b-108e-4239-8c86-da86374dbba2","Type":"ContainerStarted","Data":"6632a54715b485db31b0a1c621457bccb273802fc389922b7e25ebf0d4d04b22"}
Oct 30 21:45:39.085170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.085112    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:39.135418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.135365    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn" podStartSLOduration=2.135345645 podStartE2EDuration="2.135345645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:39.130561413 +0000 UTC m=+2721.904961353" watchObservedRunningTime="2026-10-30 21:45:39.135345645 +0000 UTC m=+2721.909745581"
Oct 30 21:45:39.179668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:39.179611    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f" podStartSLOduration=2.17959332 podStartE2EDuration="2.17959332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:39.17527465 +0000 UTC m=+2721.949674580" watchObservedRunningTime="2026-10-30 21:45:39.17959332 +0000 UTC m=+2721.953993259"
Oct 30 21:45:40.139324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:40.139293    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:41.878857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:41.871568    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:43.393915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:43.393881    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.632855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:45.631061    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:47.497674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:47.497639    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:48.944648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:48.944605    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.097814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:50.097786    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:45:50.098396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:50.098374    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:45:50.364929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:50.364857    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:51.920375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:51.920337    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:53.422524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:53.422474    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:54.854291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:54.854249    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:56.574437 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:45:56.574393    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:45:56.723583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:56.723547    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:58.225329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:58.225285    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:58.366854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:58.359390    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:59.976367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:45:59.976328    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:01.433404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:01.433361    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:02.833369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:02.833335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:04.383874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:04.383821    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:05.786489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:05.786456    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:07.120624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:07.120583    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:07.341399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:07.341363    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:08.817022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:08.816987    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:09.938262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:09.938225    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:10.285176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:10.285086    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:10.920638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:10.920582    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:11.925116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:11.925072    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:13.391659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:13.391626    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:13.539854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:13.539445    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:15.013281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:15.013240    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:16.636445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:16.636410    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:18.279511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:18.279459    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:19.817953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:19.817917    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:21.352727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:21.352692    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:22.882427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:22.882387    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:23.379417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:23.379382    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:23.500482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:23.500443    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:23.619782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:23.619748    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:24.425675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:24.425640    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:25.797058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:25.797024    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:26.575953 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:46:26.575918    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:46:27.192111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:27.192074    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:28.733792 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:28.733756    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:30.213393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:30.213361    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:31.292484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:31.292447    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:31.698728 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:31.698688    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:33.185874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:33.185791    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:35.134872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:35.134817    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:36.825515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:36.825470    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:38.314156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:38.314120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:39.876348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:39.876314    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:41.355191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:41.355150    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:43.029726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:43.029687    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:44.810259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:44.810222    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.428168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:46.428127    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:47.884027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:47.883989    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:49.339891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:49.339783    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:50.911552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:50.911512    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:52.362606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:52.362438    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:54.046185 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:54.046144    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:55.766498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:55.766468    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:56.575011 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:46:56.574974    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:46:56.918050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:56.917947    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:56.966289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:56.966250    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"]
Oct 30 21:46:56.998693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:56.998661    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"]
Oct 30 21:46:57.058429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.058398    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:57.204223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.204145    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:57.295926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.295894    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"]
Oct 30 21:46:57.300357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.300334    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.328087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.328055    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c3bbba\""
Oct 30 21:46:57.328343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.328307    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c3bbba-dockercfg-mbf2m\""
Oct 30 21:46:57.330898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.330870    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"]
Oct 30 21:46:57.472412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.471221    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8npkm\" (UniqueName: \"kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm\") pod \"payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.472412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.471310    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.520270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.520231    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:57.573221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.572841    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8npkm\" (UniqueName: \"kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm\") pod \"payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.573221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.572907    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.573948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.573918    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.686219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.686100    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8npkm\" (UniqueName: \"kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm\") pod \"payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:57.919594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:57.919550    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:46:58.126789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:58.126753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:58.242903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:58.242863    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"]
Oct 30 21:46:58.244167 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:46:58.244142    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d5093c_7ee9_4d4b_b666_f529a6bf492d.slice/crio-ca5b96e8f7571f3d3c84757a00981d0aea66d41ab9875290d4212ae43ab6e703 WatchSource:0}: Error finding container ca5b96e8f7571f3d3c84757a00981d0aea66d41ab9875290d4212ae43ab6e703: Status 404 returned error can't find the container with id ca5b96e8f7571f3d3c84757a00981d0aea66d41ab9875290d4212ae43ab6e703
Oct 30 21:46:58.246374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:58.246356    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:46:58.408413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:58.408376    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"]
Oct 30 21:46:58.479722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:58.479684    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:58.792233 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:58.792193    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl" event={"ID":"66d5093c-7ee9-4d4b-b666-f529a6bf492d","Type":"ContainerStarted","Data":"ca5b96e8f7571f3d3c84757a00981d0aea66d41ab9875290d4212ae43ab6e703"}
Oct 30 21:46:59.818663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:46:59.818636    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:00.809712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:00.809676    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl" event={"ID":"66d5093c-7ee9-4d4b-b666-f529a6bf492d","Type":"ContainerStarted","Data":"646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34"}
Oct 30 21:47:00.809958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:00.809894    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:47:00.853555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:00.853499    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl" podStartSLOduration=2.293444221 podStartE2EDuration="3.853482884s" totalImagesPullingTime="1.560038663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:00.852439483 +0000 UTC m=+2803.626839423" watchObservedRunningTime="2026-10-30 21:47:00.853482884 +0000 UTC m=+2803.627882822"
Oct 30 21:47:01.509112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:01.509044    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:01.966674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:01.966635    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f" podUID="b832b36b-108e-4239-8c86-da86374dbba2" containerName="payload-processing" containerID="cri-o://141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868" gracePeriod=25
Oct 30 21:47:01.999186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:01.999135    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn" podUID="65130ab3-f5ca-44b5-ac53-d7868b783422" containerName="payload-pre-processing" containerID="cri-o://b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f" gracePeriod=25
Oct 30 21:47:02.226278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.226178    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"]
Oct 30 21:47:02.295098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.295000    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:02.485559 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.485486    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:47:02.501456 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.501428    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:47:02.633607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.633539    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume\") pod \"b832b36b-108e-4239-8c86-da86374dbba2\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") "
Oct 30 21:47:02.633884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.633644    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw\" (UniqueName: \"kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw\") pod \"b832b36b-108e-4239-8c86-da86374dbba2\" (UID: \"b832b36b-108e-4239-8c86-da86374dbba2\") "
Oct 30 21:47:02.633884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.633678    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume\") pod \"65130ab3-f5ca-44b5-ac53-d7868b783422\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") "
Oct 30 21:47:02.633884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.633759    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2\" (UniqueName: \"kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2\") pod \"65130ab3-f5ca-44b5-ac53-d7868b783422\" (UID: \"65130ab3-f5ca-44b5-ac53-d7868b783422\") "
Oct 30 21:47:02.634099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.634064    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume" pod "b832b36b-108e-4239-8c86-da86374dbba2" (UID: "b832b36b-108e-4239-8c86-da86374dbba2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:02.634220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.634195    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume" pod "65130ab3-f5ca-44b5-ac53-d7868b783422" (UID: "65130ab3-f5ca-44b5-ac53-d7868b783422"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:02.634355 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.634316    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b832b36b-108e-4239-8c86-da86374dbba2-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:02.636106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.636083    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw" pod "b832b36b-108e-4239-8c86-da86374dbba2" (UID: "b832b36b-108e-4239-8c86-da86374dbba2"). InnerVolumeSpecName "kube-api-access-vc9kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:02.636201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.636127    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2" pod "65130ab3-f5ca-44b5-ac53-d7868b783422" (UID: "65130ab3-f5ca-44b5-ac53-d7868b783422"). InnerVolumeSpecName "kube-api-access-qhnn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:02.736231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.736130    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhnn2\" (UniqueName: \"kubernetes.io/projected/65130ab3-f5ca-44b5-ac53-d7868b783422-kube-api-access-qhnn2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:02.736231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.736166    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc9kw\" (UniqueName: \"kubernetes.io/projected/b832b36b-108e-4239-8c86-da86374dbba2-kube-api-access-vc9kw\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:02.736231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.736177    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65130ab3-f5ca-44b5-ac53-d7868b783422-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:02.827332 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.827284    4549 generic.go:356] "Generic (PLEG): container finished" podID="65130ab3-f5ca-44b5-ac53-d7868b783422" containerID="b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f" exitCode=0
Oct 30 21:47:02.827537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.827350    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"
Oct 30 21:47:02.827537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.827357    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn" event={"ID":"65130ab3-f5ca-44b5-ac53-d7868b783422","Type":"ContainerDied","Data":"b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f"}
Oct 30 21:47:02.827537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.827445    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn" event={"ID":"65130ab3-f5ca-44b5-ac53-d7868b783422","Type":"ContainerDied","Data":"5df9f530e9b83319f28cd8d3f7f52324afde2805ea8237218c03bed6c409973a"}
Oct 30 21:47:02.827537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.827476    4549 scope.go:122] "RemoveContainer" containerID="b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f"
Oct 30 21:47:02.829217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.829184    4549 generic.go:356] "Generic (PLEG): container finished" podID="b832b36b-108e-4239-8c86-da86374dbba2" containerID="141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868" exitCode=0
Oct 30 21:47:02.829340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.829289    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f" event={"ID":"b832b36b-108e-4239-8c86-da86374dbba2","Type":"ContainerDied","Data":"141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868"}
Oct 30 21:47:02.829340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.829335    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f" event={"ID":"b832b36b-108e-4239-8c86-da86374dbba2","Type":"ContainerDied","Data":"6632a54715b485db31b0a1c621457bccb273802fc389922b7e25ebf0d4d04b22"}
Oct 30 21:47:02.829440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.829342    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"
Oct 30 21:47:02.840734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.840715    4549 scope.go:122] "RemoveContainer" containerID="b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f"
Oct 30 21:47:02.841106 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:02.841086    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f\": container with ID starting with b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f not found: ID does not exist" containerID="b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f"
Oct 30 21:47:02.841201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.841114    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f"} err="failed to get container status \"b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f\": rpc error: code = NotFound desc = could not find container \"b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f\": container with ID starting with b22583b6dd8322044ecbe1b19301a835d5128b6554945db606e649fa67d75a2f not found: ID does not exist"
Oct 30 21:47:02.841201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.841130    4549 scope.go:122] "RemoveContainer" containerID="141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868"
Oct 30 21:47:02.851755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.851731    4549 scope.go:122] "RemoveContainer" containerID="141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868"
Oct 30 21:47:02.852193 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:02.852155    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868\": container with ID starting with 141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868 not found: ID does not exist" containerID="141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868"
Oct 30 21:47:02.852264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.852208    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868"} err="failed to get container status \"141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868\": rpc error: code = NotFound desc = could not find container \"141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868\": container with ID starting with 141f8a71bdda5bd9b3e9428061dd421cf8388cc74734a8c22d33573579450868 not found: ID does not exist"
Oct 30 21:47:02.887632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.887597    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"]
Oct 30 21:47:02.894977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.894934    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:02.899457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.899425    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn"]
Oct 30 21:47:02.924258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.924219    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"]
Oct 30 21:47:02.927724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:02.927695    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f"]
Oct 30 21:47:03.234900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.234864    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:03.300648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.300611    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"]
Oct 30 21:47:03.301731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.301705    4549 cpu_manager.go:414] "containerMap: removing container" podUID="65130ab3-f5ca-44b5-ac53-d7868b783422" containerName="payload-pre-processing"
Oct 30 21:47:03.301731 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.301731    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65130ab3-f5ca-44b5-ac53-d7868b783422" containerName="payload-pre-processing"
Oct 30 21:47:03.301976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.301776    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b832b36b-108e-4239-8c86-da86374dbba2" containerName="payload-processing"
Oct 30 21:47:03.301976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.301786    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b832b36b-108e-4239-8c86-da86374dbba2" containerName="payload-processing"
Oct 30 21:47:03.301976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.301974    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="65130ab3-f5ca-44b5-ac53-d7868b783422" containerName="payload-pre-processing"
Oct 30 21:47:03.302146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.301991    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b832b36b-108e-4239-8c86-da86374dbba2" containerName="payload-processing"
Oct 30 21:47:03.307880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.307851    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.331173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.331139    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-e1d376-openshift-default-dockercfg-57h22\""
Oct 30 21:47:03.334563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.334532    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"]
Oct 30 21:47:03.349033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.348982    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65130ab3-f5ca-44b5-ac53-d7868b783422" path="/var/lib/kubelet/pods/65130ab3-f5ca-44b5-ac53-d7868b783422/volumes"
Oct 30 21:47:03.349594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.349570    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b832b36b-108e-4239-8c86-da86374dbba2" path="/var/lib/kubelet/pods/b832b36b-108e-4239-8c86-da86374dbba2/volumes"
Oct 30 21:47:03.351621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.351554    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352068    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352285    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6zkw\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352427    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352577    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352636    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352791    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352823    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.352943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.352898    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.488138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.488031    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.492158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.492117    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6zkw\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.506992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.501369    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:03.507439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.507414    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.508343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.508318    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.515015    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.515206    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.515233    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.515295    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.515426    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.515943    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.521865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.509308    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.529902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.523735    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.529902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.524062    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.529902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.524301    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.529902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.524932    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.556858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.540600    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6zkw\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.572852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.570062    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.579764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.579474    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token\") pod \"e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") " pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.632358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.632324    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:03.939486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.939098    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:03.962140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.962111    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"]
Oct 30 21:47:03.963168 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:03.963136    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c031ff_19b0_42ba_9ba3_f9049f270dbb.slice/crio-2ebec0ad910f0ef00b5db6e0ed8aca296a2052e7854cd9a37557ff5732c7f005 WatchSource:0}: Error finding container 2ebec0ad910f0ef00b5db6e0ed8aca296a2052e7854cd9a37557ff5732c7f005: Status 404 returned error can't find the container with id 2ebec0ad910f0ef00b5db6e0ed8aca296a2052e7854cd9a37557ff5732c7f005
Oct 30 21:47:03.966452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.966417    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:03.966558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.966500    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:03.966558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:03.966548    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:04.740998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:04.732944    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:04.847348 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:04.847309    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" event={"ID":"71c031ff-19b0-42ba-9ba3-f9049f270dbb","Type":"ContainerStarted","Data":"c906afc1ca3ebdba8d04b12523881b6ce5c1b5cf73b21bbc15193179780c7178"}
Oct 30 21:47:04.847565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:04.847367    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" event={"ID":"71c031ff-19b0-42ba-9ba3-f9049f270dbb","Type":"ContainerStarted","Data":"2ebec0ad910f0ef00b5db6e0ed8aca296a2052e7854cd9a37557ff5732c7f005"}
Oct 30 21:47:04.896670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:04.896611    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podStartSLOduration=1.89659421 podStartE2EDuration="1.89659421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:04.892241606 +0000 UTC m=+2807.666641545" watchObservedRunningTime="2026-10-30 21:47:04.89659421 +0000 UTC m=+2807.670994148"
Oct 30 21:47:05.633028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:05.632983    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:05.640179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:05.640146    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:05.853715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:05.853680    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:05.854900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:05.854882    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:47:06.487997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:06.487963    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:07.378878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:07.378841    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:07.560454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:07.560408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:08.431486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:08.431454    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:08.996492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:08.996462    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:09.135209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.135173    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"]
Oct 30 21:47:09.136313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.136283    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:09.139879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.139816    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.157540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.157509    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"]
Oct 30 21:47:09.165267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.165205    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e1d376\""
Oct 30 21:47:09.165267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.165212    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e1d376-dockercfg-hgsg9\""
Oct 30 21:47:09.331857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.327576    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.331857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.327735    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf2v2\" (UniqueName: \"kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.397503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.397468    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"]
Oct 30 21:47:09.404385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.403610    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.408952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.408919    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"]
Oct 30 21:47:09.429658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.429617    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.429894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.429720    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2566b\" (UniqueName: \"kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b\") pod \"payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.429894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.429778    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.430028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.429916    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf2v2\" (UniqueName: \"kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.433205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.431600    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.450795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.450758    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf2v2\" (UniqueName: \"kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.455627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.455599    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:09.535425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.535390    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2566b\" (UniqueName: \"kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b\") pod \"payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.535589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.535437    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.536296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.536267    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.555978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.555943    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2566b\" (UniqueName: \"kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b\") pod \"payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.724240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.724146    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:09.771242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.771204    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"]
Oct 30 21:47:09.780011 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:09.779971    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c71507f_77c9_4017_8d88_9ef13130ec04.slice/crio-dc4f8535c9917617b705ed0dd24c62759d40325f2ccd40c1318425d6631cc43a WatchSource:0}: Error finding container dc4f8535c9917617b705ed0dd24c62759d40325f2ccd40c1318425d6631cc43a: Status 404 returned error can't find the container with id dc4f8535c9917617b705ed0dd24c62759d40325f2ccd40c1318425d6631cc43a
Oct 30 21:47:09.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:09.887887    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls" event={"ID":"7c71507f-77c9-4017-8d88-9ef13130ec04","Type":"ContainerStarted","Data":"dc4f8535c9917617b705ed0dd24c62759d40325f2ccd40c1318425d6631cc43a"}
Oct 30 21:47:10.047858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.047797    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"]
Oct 30 21:47:10.049670 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:10.049639    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba21ca0_ef35_43b0_91df_bb17138968bf.slice/crio-f33b38e1062a7aac00e436742b43132203d1919d3773c452bdf2d4f70916c0bd WatchSource:0}: Error finding container f33b38e1062a7aac00e436742b43132203d1919d3773c452bdf2d4f70916c0bd: Status 404 returned error can't find the container with id f33b38e1062a7aac00e436742b43132203d1919d3773c452bdf2d4f70916c0bd
Oct 30 21:47:10.109545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.109507    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:10.248621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.248570    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:10.896947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.896905    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz" event={"ID":"3ba21ca0-ef35-43b0-91df-bb17138968bf","Type":"ContainerStarted","Data":"1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658"}
Oct 30 21:47:10.897406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.896979    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:10.897406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.897005    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz" event={"ID":"3ba21ca0-ef35-43b0-91df-bb17138968bf","Type":"ContainerStarted","Data":"f33b38e1062a7aac00e436742b43132203d1919d3773c452bdf2d4f70916c0bd"}
Oct 30 21:47:10.898678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.898642    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls" event={"ID":"7c71507f-77c9-4017-8d88-9ef13130ec04","Type":"ContainerStarted","Data":"42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc"}
Oct 30 21:47:10.898819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.898722    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:10.945054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:10.944737    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz" podStartSLOduration=1.944717877 podStartE2EDuration="1.944717877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:10.942917626 +0000 UTC m=+2813.717317566" watchObservedRunningTime="2026-10-30 21:47:10.944717877 +0000 UTC m=+2813.719117815"
Oct 30 21:47:11.659084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:11.659054    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:11.820871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:11.820817    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:47:11.871557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:11.871500    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls" podStartSLOduration=2.871485418 podStartE2EDuration="2.871485418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:10.99209959 +0000 UTC m=+2813.766499526" watchObservedRunningTime="2026-10-30 21:47:11.871485418 +0000 UTC m=+2814.645885403"
Oct 30 21:47:13.123645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.123599    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:13.434190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.434095    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k" podUID="c842686b-2734-4d66-9b46-253dadde14c8" containerName="main" containerID="cri-o://07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5" gracePeriod=45
Oct 30 21:47:13.709419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.709388    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:47:13.796215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796173    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir\") pod \"c842686b-2734-4d66-9b46-253dadde14c8\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") "
Oct 30 21:47:13.796398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796293    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j\" (UniqueName: \"kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j\") pod \"c842686b-2734-4d66-9b46-253dadde14c8\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") "
Oct 30 21:47:13.796462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796445    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm\") pod \"c842686b-2734-4d66-9b46-253dadde14c8\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") "
Oct 30 21:47:13.796542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796501    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir" pod "c842686b-2734-4d66-9b46-253dadde14c8" (UID: "c842686b-2734-4d66-9b46-253dadde14c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:13.796631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796549    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs\" (UniqueName: \"kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs\") pod \"c842686b-2734-4d66-9b46-253dadde14c8\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") "
Oct 30 21:47:13.796631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796586    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache\") pod \"c842686b-2734-4d66-9b46-253dadde14c8\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") "
Oct 30 21:47:13.796631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796617    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home\") pod \"c842686b-2734-4d66-9b46-253dadde14c8\" (UID: \"c842686b-2734-4d66-9b46-253dadde14c8\") "
Oct 30 21:47:13.796921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796879    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache" pod "c842686b-2734-4d66-9b46-253dadde14c8" (UID: "c842686b-2734-4d66-9b46-253dadde14c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:13.797018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.796983    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home" pod "c842686b-2734-4d66-9b46-253dadde14c8" (UID: "c842686b-2734-4d66-9b46-253dadde14c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:13.797385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.797360    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:13.797385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.797387    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:13.797562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.797400    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:13.799067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.799045    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j" pod "c842686b-2734-4d66-9b46-253dadde14c8" (UID: "c842686b-2734-4d66-9b46-253dadde14c8"). InnerVolumeSpecName "kube-api-access-tbf6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:13.799330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.799308    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs" pod "c842686b-2734-4d66-9b46-253dadde14c8" (UID: "c842686b-2734-4d66-9b46-253dadde14c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:13.799330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.799305    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm" pod "c842686b-2734-4d66-9b46-253dadde14c8" (UID: "c842686b-2734-4d66-9b46-253dadde14c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:13.898817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.898777    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c842686b-2734-4d66-9b46-253dadde14c8-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:13.898817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.898810    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c842686b-2734-4d66-9b46-253dadde14c8-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:13.898817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.898820    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbf6j\" (UniqueName: \"kubernetes.io/projected/c842686b-2734-4d66-9b46-253dadde14c8-kube-api-access-tbf6j\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:13.921895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.921851    4549 generic.go:356] "Generic (PLEG): container finished" podID="c842686b-2734-4d66-9b46-253dadde14c8" containerID="07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5" exitCode=0
Oct 30 21:47:13.921895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.921893    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"
Oct 30 21:47:13.922115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.921937    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k" event={"ID":"c842686b-2734-4d66-9b46-253dadde14c8","Type":"ContainerDied","Data":"07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5"}
Oct 30 21:47:13.922115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.921981    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k" event={"ID":"c842686b-2734-4d66-9b46-253dadde14c8","Type":"ContainerDied","Data":"3e4dc562eacd8bf10f13b2b64a5b23de1a478b88e8486826f02a89e9dacf5f1b"}
Oct 30 21:47:13.922115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.922002    4549 scope.go:122] "RemoveContainer" containerID="07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5"
Oct 30 21:47:13.934080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.934058    4549 scope.go:122] "RemoveContainer" containerID="07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5"
Oct 30 21:47:13.934386 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:13.934368    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5\": container with ID starting with 07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5 not found: ID does not exist" containerID="07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5"
Oct 30 21:47:13.934444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.934395    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5"} err="failed to get container status \"07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5\": rpc error: code = NotFound desc = could not find container \"07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5\": container with ID starting with 07c8dad65f5e617281c384fbfbdfe0c7fca9fb9f7d3573e2da5ea9a6b66379b5 not found: ID does not exist"
Oct 30 21:47:13.981393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.981297    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"]
Oct 30 21:47:13.989303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:13.989259    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ad73a4/rate-test-model-ad73a4-kserve-886bdfdb9-w578k"]
Oct 30 21:47:14.959692 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:14.958275    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:15.349505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:15.349470    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c842686b-2734-4d66-9b46-253dadde14c8" path="/var/lib/kubelet/pods/c842686b-2734-4d66-9b46-253dadde14c8/volumes"
Oct 30 21:47:16.446901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:16.446864    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"]
Oct 30 21:47:16.524863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:16.524813    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:16.977886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:16.977848    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:17.256353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.256259    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l" podUID="fc1797d3-96e0-410e-b523-f6a1975ca35a" containerName="main" containerID="cri-o://744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947" gracePeriod=45
Oct 30 21:47:17.559917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.559888    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:47:17.661981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.661946    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache\") pod \"fc1797d3-96e0-410e-b523-f6a1975ca35a\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") "
Oct 30 21:47:17.662158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662086    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home\") pod \"fc1797d3-96e0-410e-b523-f6a1975ca35a\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") "
Oct 30 21:47:17.662215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662155    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs\") pod \"fc1797d3-96e0-410e-b523-f6a1975ca35a\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") "
Oct 30 21:47:17.662256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662217    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm\") pod \"fc1797d3-96e0-410e-b523-f6a1975ca35a\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") "
Oct 30 21:47:17.662312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662239    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache" pod "fc1797d3-96e0-410e-b523-f6a1975ca35a" (UID: "fc1797d3-96e0-410e-b523-f6a1975ca35a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:17.662312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662248    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg\" (UniqueName: \"kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg\") pod \"fc1797d3-96e0-410e-b523-f6a1975ca35a\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") "
Oct 30 21:47:17.662429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662355    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir\") pod \"fc1797d3-96e0-410e-b523-f6a1975ca35a\" (UID: \"fc1797d3-96e0-410e-b523-f6a1975ca35a\") "
Oct 30 21:47:17.662429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662389    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home" pod "fc1797d3-96e0-410e-b523-f6a1975ca35a" (UID: "fc1797d3-96e0-410e-b523-f6a1975ca35a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:17.662651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.662621    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir" pod "fc1797d3-96e0-410e-b523-f6a1975ca35a" (UID: "fc1797d3-96e0-410e-b523-f6a1975ca35a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:17.663263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.663237    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:17.663345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.663269    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:17.663345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.663310    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:17.664495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.664466    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm" pod "fc1797d3-96e0-410e-b523-f6a1975ca35a" (UID: "fc1797d3-96e0-410e-b523-f6a1975ca35a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:17.664585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.664514    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs" pod "fc1797d3-96e0-410e-b523-f6a1975ca35a" (UID: "fc1797d3-96e0-410e-b523-f6a1975ca35a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:17.664638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.664600    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg" pod "fc1797d3-96e0-410e-b523-f6a1975ca35a" (UID: "fc1797d3-96e0-410e-b523-f6a1975ca35a"). InnerVolumeSpecName "kube-api-access-jf9wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:17.764761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.764719    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc1797d3-96e0-410e-b523-f6a1975ca35a-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:17.764761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.764752    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc1797d3-96e0-410e-b523-f6a1975ca35a-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:17.764761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.764765    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf9wg\" (UniqueName: \"kubernetes.io/projected/fc1797d3-96e0-410e-b523-f6a1975ca35a-kube-api-access-jf9wg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:17.997849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.997792    4549 generic.go:356] "Generic (PLEG): container finished" podID="fc1797d3-96e0-410e-b523-f6a1975ca35a" containerID="744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947" exitCode=0
Oct 30 21:47:17.998046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.997853    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"
Oct 30 21:47:17.998046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.997864    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l" event={"ID":"fc1797d3-96e0-410e-b523-f6a1975ca35a","Type":"ContainerDied","Data":"744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947"}
Oct 30 21:47:17.998046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.997913    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l" event={"ID":"fc1797d3-96e0-410e-b523-f6a1975ca35a","Type":"ContainerDied","Data":"3088b6295db25486a7af3b6d0ff769819c2a71955058eba4f4117650f12f7f2a"}
Oct 30 21:47:17.998046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:17.997935    4549 scope.go:122] "RemoveContainer" containerID="744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947"
Oct 30 21:47:18.009291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:18.009267    4549 scope.go:122] "RemoveContainer" containerID="744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947"
Oct 30 21:47:18.009568 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:18.009547    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947\": container with ID starting with 744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947 not found: ID does not exist" containerID="744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947"
Oct 30 21:47:18.009637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:18.009582    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947"} err="failed to get container status \"744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947\": rpc error: code = NotFound desc = could not find container \"744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947\": container with ID starting with 744178ab40a38bc118c72da9e817b43f0c786b21d501c5cefeab59e86bbe6947 not found: ID does not exist"
Oct 30 21:47:18.050680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:18.050644    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"]
Oct 30 21:47:18.062425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:18.062392    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-70d3a9/rate-test-model-70d3a9-kserve-85b885599c-pk59l"]
Oct 30 21:47:18.544453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:18.544413    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:19.345747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:19.345698    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc1797d3-96e0-410e-b523-f6a1975ca35a" path="/var/lib/kubelet/pods/fc1797d3-96e0-410e-b523-f6a1975ca35a/volumes"
Oct 30 21:47:20.115572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:20.115533    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:21.627406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:21.627371    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:21.910659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:21.910577    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:47:21.910659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:21.910643    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:47:23.207785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:23.207748    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:24.868897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:24.868720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:26.324140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:26.324107    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:26.576872 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:26.576760    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:47:27.896408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:27.896373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:29.459220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:29.459126    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:31.031360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.031323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:31.472519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.472471    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt" podUID="072a5017-9232-45bd-b22e-ce7cda4042ff" containerName="main" containerID="cri-o://84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1" gracePeriod=45
Oct 30 21:47:31.747144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.747113    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:47:31.867745 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.867706    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t\" (UniqueName: \"kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t\") pod \"072a5017-9232-45bd-b22e-ce7cda4042ff\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") "
Oct 30 21:47:31.867991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.867765    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache\") pod \"072a5017-9232-45bd-b22e-ce7cda4042ff\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") "
Oct 30 21:47:31.867991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.867869    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home\") pod \"072a5017-9232-45bd-b22e-ce7cda4042ff\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") "
Oct 30 21:47:31.867991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.867944    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm\") pod \"072a5017-9232-45bd-b22e-ce7cda4042ff\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") "
Oct 30 21:47:31.868272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868022    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir\") pod \"072a5017-9232-45bd-b22e-ce7cda4042ff\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") "
Oct 30 21:47:31.868272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868051    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs\" (UniqueName: \"kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs\") pod \"072a5017-9232-45bd-b22e-ce7cda4042ff\" (UID: \"072a5017-9232-45bd-b22e-ce7cda4042ff\") "
Oct 30 21:47:31.868272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868195    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home" pod "072a5017-9232-45bd-b22e-ce7cda4042ff" (UID: "072a5017-9232-45bd-b22e-ce7cda4042ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:31.868272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868223    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache" pod "072a5017-9232-45bd-b22e-ce7cda4042ff" (UID: "072a5017-9232-45bd-b22e-ce7cda4042ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:31.868620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868595    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir" pod "072a5017-9232-45bd-b22e-ce7cda4042ff" (UID: "072a5017-9232-45bd-b22e-ce7cda4042ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:31.868905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868717    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:31.868905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868737    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:31.868905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.868752    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:31.870618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.870590    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t" pod "072a5017-9232-45bd-b22e-ce7cda4042ff" (UID: "072a5017-9232-45bd-b22e-ce7cda4042ff"). InnerVolumeSpecName "kube-api-access-q669t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:31.870721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.870668    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm" pod "072a5017-9232-45bd-b22e-ce7cda4042ff" (UID: "072a5017-9232-45bd-b22e-ce7cda4042ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:31.870948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.870926    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs" pod "072a5017-9232-45bd-b22e-ce7cda4042ff" (UID: "072a5017-9232-45bd-b22e-ce7cda4042ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:31.970266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.970224    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072a5017-9232-45bd-b22e-ce7cda4042ff-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:31.970266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.970261    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072a5017-9232-45bd-b22e-ce7cda4042ff-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:31.970266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:31.970276    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q669t\" (UniqueName: \"kubernetes.io/projected/072a5017-9232-45bd-b22e-ce7cda4042ff-kube-api-access-q669t\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:47:32.118552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.118515    4549 generic.go:356] "Generic (PLEG): container finished" podID="072a5017-9232-45bd-b22e-ce7cda4042ff" containerID="84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1" exitCode=0
Oct 30 21:47:32.119088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.118557    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"
Oct 30 21:47:32.119088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.118571    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt" event={"ID":"072a5017-9232-45bd-b22e-ce7cda4042ff","Type":"ContainerDied","Data":"84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1"}
Oct 30 21:47:32.119088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.118605    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt" event={"ID":"072a5017-9232-45bd-b22e-ce7cda4042ff","Type":"ContainerDied","Data":"fa7d421c5b06c4aa33d786754475d7607085aa4d7bf2a6cde0c6d604a1a521f5"}
Oct 30 21:47:32.119088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.118621    4549 scope.go:122] "RemoveContainer" containerID="84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1"
Oct 30 21:47:32.131296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.131272    4549 scope.go:122] "RemoveContainer" containerID="84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1"
Oct 30 21:47:32.131580 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:32.131558    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1\": container with ID starting with 84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1 not found: ID does not exist" containerID="84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1"
Oct 30 21:47:32.131635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.131591    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1"} err="failed to get container status \"84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1\": rpc error: code = NotFound desc = could not find container \"84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1\": container with ID starting with 84e6e2da4fe0ac4713b2076e036daee1a01e11acdcf696d0376e2431f93e66f1 not found: ID does not exist"
Oct 30 21:47:32.169894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.169859    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"]
Oct 30 21:47:32.181324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.181295    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-f5bef0/auth-test-model-f5bef0-kserve-c6dc9dff7-72vbt"]
Oct 30 21:47:32.521364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:32.521275    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:33.345172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:33.345131    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="072a5017-9232-45bd-b22e-ce7cda4042ff" path="/var/lib/kubelet/pods/072a5017-9232-45bd-b22e-ce7cda4042ff/volumes"
Oct 30 21:47:33.562475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:33.562437    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:34.126535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:34.126499    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:35.220590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.220540    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:35.394440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.393480    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:35.672857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.651602    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"]
Oct 30 21:47:35.674227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.674194    4549 cpu_manager.go:414] "containerMap: removing container" podUID="072a5017-9232-45bd-b22e-ce7cda4042ff" containerName="main"
Oct 30 21:47:35.674411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.674395    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="072a5017-9232-45bd-b22e-ce7cda4042ff" containerName="main"
Oct 30 21:47:35.674530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.674519    4549 cpu_manager.go:414] "containerMap: removing container" podUID="fc1797d3-96e0-410e-b523-f6a1975ca35a" containerName="main"
Oct 30 21:47:35.674611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.674601    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc1797d3-96e0-410e-b523-f6a1975ca35a" containerName="main"
Oct 30 21:47:35.678905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.674716    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c842686b-2734-4d66-9b46-253dadde14c8" containerName="main"
Oct 30 21:47:35.679093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.679071    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c842686b-2734-4d66-9b46-253dadde14c8" containerName="main"
Oct 30 21:47:35.679422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.679404    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c842686b-2734-4d66-9b46-253dadde14c8" containerName="main"
Oct 30 21:47:35.679555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.679543    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="072a5017-9232-45bd-b22e-ce7cda4042ff" containerName="main"
Oct 30 21:47:35.679652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.679643    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc1797d3-96e0-410e-b523-f6a1975ca35a" containerName="main"
Oct 30 21:47:35.690584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.687013    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"]
Oct 30 21:47:35.690584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.687124    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.698488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.698154    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wflp\" (UniqueName: \"kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp\") pod \"payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.698488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.698318    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.703151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.702481    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-203209\""
Oct 30 21:47:35.713780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.713733    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-203209-dockercfg-lxpsq\""
Oct 30 21:47:35.764887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.764856    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"]
Oct 30 21:47:35.770110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.770089    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:35.810853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.788114    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"]
Oct 30 21:47:35.814861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.812361    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wflp\" (UniqueName: \"kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp\") pod \"payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.815815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.815782    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.820764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.820732    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.837536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.837506    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wflp\" (UniqueName: \"kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp\") pod \"payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:35.916686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.916643    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:35.916955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:35.916774    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcqzl\" (UniqueName: \"kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl\") pod \"payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:36.017966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.017866    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:36.018151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.017987    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcqzl\" (UniqueName: \"kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl\") pod \"payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:36.018582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.018560    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:36.028910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.028883    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:36.042084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.042056    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcqzl\" (UniqueName: \"kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl\") pod \"payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:36.085584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.085544    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:36.233271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.233225    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:36.351471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.351445    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"]
Oct 30 21:47:36.353128 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:36.353104    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6292ef_6847_4ec0_9f25_52e6530dcdd5.slice/crio-7963d83dbca5a407f41b40005429b94e2e6f14664137de1d526849a0813144b5 WatchSource:0}: Error finding container 7963d83dbca5a407f41b40005429b94e2e6f14664137de1d526849a0813144b5: Status 404 returned error can't find the container with id 7963d83dbca5a407f41b40005429b94e2e6f14664137de1d526849a0813144b5
Oct 30 21:47:36.427648 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:36.427613    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"]
Oct 30 21:47:36.427869 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:36.427806    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda198c7a6_5911_4d40_aa9d_4877f4d67ad1.slice/crio-2245753fe639bdbc2389f6cb0d6fd73e3b8920d9d1ec35793282a239ff97ed37 WatchSource:0}: Error finding container 2245753fe639bdbc2389f6cb0d6fd73e3b8920d9d1ec35793282a239ff97ed37: Status 404 returned error can't find the container with id 2245753fe639bdbc2389f6cb0d6fd73e3b8920d9d1ec35793282a239ff97ed37
Oct 30 21:47:37.165232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.165190    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b" event={"ID":"a198c7a6-5911-4d40-aa9d-4877f4d67ad1","Type":"ContainerStarted","Data":"6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b"}
Oct 30 21:47:37.165511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.165482    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b" event={"ID":"a198c7a6-5911-4d40-aa9d-4877f4d67ad1","Type":"ContainerStarted","Data":"2245753fe639bdbc2389f6cb0d6fd73e3b8920d9d1ec35793282a239ff97ed37"}
Oct 30 21:47:37.165660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.165556    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:37.167131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.167104    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn" event={"ID":"6d6292ef-6847-4ec0-9f25-52e6530dcdd5","Type":"ContainerStarted","Data":"b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5"}
Oct 30 21:47:37.167284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.167149    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn" event={"ID":"6d6292ef-6847-4ec0-9f25-52e6530dcdd5","Type":"ContainerStarted","Data":"7963d83dbca5a407f41b40005429b94e2e6f14664137de1d526849a0813144b5"}
Oct 30 21:47:37.167284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.167178    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:37.207460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.207402    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b" podStartSLOduration=2.20738841 podStartE2EDuration="2.20738841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:37.205800565 +0000 UTC m=+2839.980200507" watchObservedRunningTime="2026-10-30 21:47:37.20738841 +0000 UTC m=+2839.981788345"
Oct 30 21:47:37.251196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.251132    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn" podStartSLOduration=2.251112747 podStartE2EDuration="2.251112747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:37.247790088 +0000 UTC m=+2840.022190029" watchObservedRunningTime="2026-10-30 21:47:37.251112747 +0000 UTC m=+2840.025512689"
Oct 30 21:47:37.876116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:37.876078    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:39.815751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:39.815697    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:41.332447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:41.332414    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:43.236480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:43.236448    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:44.842010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:44.841973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:46.420743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:46.420703    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:47.972739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:47.972699    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.142865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:48.139121    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.179111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:48.179082    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:47:48.180028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:48.180001    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:47:48.348121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:48.348086    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:49.781747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:49.781715    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:49.933629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:49.933576    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:50.350115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:50.350074    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:50.521193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:50.521153    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:51.402291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:51.402249    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:53.054324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:53.054291    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:53.983443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:53.983404    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:54.128367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:54.128331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:54.403317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:54.403270    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:54.804732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:54.804622    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:55.646984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.646944    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:55.822973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.822941    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"]
Oct 30 21:47:55.826544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.826510    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:55.827450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.827432    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:55.843240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.843214    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e60e16\""
Oct 30 21:47:55.855339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.855311    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e60e16-dockercfg-s7zpr\""
Oct 30 21:47:55.856318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.856292    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24ltk\" (UniqueName: \"kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk\") pod \"payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:55.856620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.856593    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:55.860761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.860737    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"]
Oct 30 21:47:55.981921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.963296    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24ltk\" (UniqueName: \"kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk\") pod \"payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:55.981921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.963441    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:55.992210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:55.988598    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:56.000269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.000230    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"]
Oct 30 21:47:56.030197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.030157    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"]
Oct 30 21:47:56.030398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.030290    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.069209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.068649    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24ltk\" (UniqueName: \"kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk\") pod \"payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:56.143072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.143034    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:56.156998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.156965    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.157210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.157186    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjpb\" (UniqueName: \"kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb\") pod \"payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.259183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.259082    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.259363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.259256    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjpb\" (UniqueName: \"kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb\") pod \"payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.259899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.259873    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.281691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.281649    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjpb\" (UniqueName: \"kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb\") pod \"payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.397114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.397080    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:56.473384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.473350    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"]
Oct 30 21:47:56.480630 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:56.480593    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1ce0c3_9fa2_4bdb_9058_599d11e40578.slice/crio-f27d1417cab0dfdb2060dc4b6bd0fd313c9c54063f980b922129c019e3417fa1 WatchSource:0}: Error finding container f27d1417cab0dfdb2060dc4b6bd0fd313c9c54063f980b922129c019e3417fa1: Status 404 returned error can't find the container with id f27d1417cab0dfdb2060dc4b6bd0fd313c9c54063f980b922129c019e3417fa1
Oct 30 21:47:56.581968 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:47:56.581941    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:47:56.643727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.641875    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:56.725999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:56.725973    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"]
Oct 30 21:47:56.726612 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:47:56.726591    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e44ccc_6631_48dd_b2b5_4576ae8eb674.slice/crio-afc25e77ca5372f05669a54ee7db7731a4985251696d1f6e17ed9bf973a0c462 WatchSource:0}: Error finding container afc25e77ca5372f05669a54ee7db7731a4985251696d1f6e17ed9bf973a0c462: Status 404 returned error can't find the container with id afc25e77ca5372f05669a54ee7db7731a4985251696d1f6e17ed9bf973a0c462
Oct 30 21:47:57.334942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.334903    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq" event={"ID":"39e44ccc-6631-48dd-b2b5-4576ae8eb674","Type":"ContainerStarted","Data":"a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650"}
Oct 30 21:47:57.335150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.334970    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq" event={"ID":"39e44ccc-6631-48dd-b2b5-4576ae8eb674","Type":"ContainerStarted","Data":"afc25e77ca5372f05669a54ee7db7731a4985251696d1f6e17ed9bf973a0c462"}
Oct 30 21:47:57.335150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.335010    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:47:57.344588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.344562    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:47:57.344734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.344609    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5" event={"ID":"9e1ce0c3-9fa2-4bdb-9058-599d11e40578","Type":"ContainerStarted","Data":"9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef"}
Oct 30 21:47:57.344734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.344633    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5" event={"ID":"9e1ce0c3-9fa2-4bdb-9058-599d11e40578","Type":"ContainerStarted","Data":"f27d1417cab0dfdb2060dc4b6bd0fd313c9c54063f980b922129c019e3417fa1"}
Oct 30 21:47:57.377334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.377270    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq" podStartSLOduration=2.377250518 podStartE2EDuration="2.377250518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:57.374164665 +0000 UTC m=+2860.148564605" watchObservedRunningTime="2026-10-30 21:47:57.377250518 +0000 UTC m=+2860.151650459"
Oct 30 21:47:57.422122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.422067    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5" podStartSLOduration=2.422053112 podStartE2EDuration="2.422053112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:57.417208936 +0000 UTC m=+2860.191608873" watchObservedRunningTime="2026-10-30 21:47:57.422053112 +0000 UTC m=+2860.196453051"
Oct 30 21:47:57.973777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:57.973740    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.130116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:58.130079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.272961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:58.272806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.513935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:58.511032    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.684489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:58.684424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.898862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:58.898796    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:59.147207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:47:59.147162    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:00.243125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:00.243086    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:01.766330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:01.766294    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:03.244299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:03.244266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:04.929480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:04.929439    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:05.877738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:05.877686    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.035712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:06.035653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.189106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:06.189025    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"]
Oct 30 21:48:06.189341 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:06.189310    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy" containerID="cri-o://de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7" gracePeriod=30
Oct 30 21:48:06.276552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:06.276517    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.441812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:06.441712    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.609541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:06.609433    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:08.225099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:08.225065    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:08.407446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:08.407418    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:48:08.407643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:08.407489    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:48:09.726923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:09.726886    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:10.215012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:10.214973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:11.141013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.140979    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:11.487568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.487530    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:48:11.505356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.505327    4549 generic.go:356] "Generic (PLEG): container finished" podID="83daaa67-1780-4025-8921-3466c3867f59" containerID="de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7" exitCode=0
Oct 30 21:48:11.505536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.505373    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"
Oct 30 21:48:11.505536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.505410    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" event={"ID":"83daaa67-1780-4025-8921-3466c3867f59","Type":"ContainerDied","Data":"de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7"}
Oct 30 21:48:11.505536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.505466    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx" event={"ID":"83daaa67-1780-4025-8921-3466c3867f59","Type":"ContainerDied","Data":"e76767d3bf2daac0a0de854eeab5c53b97746a8e061c940409ec1a3993abba6b"}
Oct 30 21:48:11.505536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.505489    4549 scope.go:122] "RemoveContainer" containerID="de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7"
Oct 30 21:48:11.516485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.516462    4549 scope.go:122] "RemoveContainer" containerID="a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"
Oct 30 21:48:11.528272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.528248    4549 scope.go:122] "RemoveContainer" containerID="de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7"
Oct 30 21:48:11.529531 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:11.529504    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7\": container with ID starting with de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7 not found: ID does not exist" containerID="de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7"
Oct 30 21:48:11.529677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.529539    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7"} err="failed to get container status \"de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7\": rpc error: code = NotFound desc = could not find container \"de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7\": container with ID starting with de3a281768d97ce77079760a6749255b5dc492af286a247565691d292fdd6ae7 not found: ID does not exist"
Oct 30 21:48:11.529677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.529558    4549 scope.go:122] "RemoveContainer" containerID="a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"
Oct 30 21:48:11.529907 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:11.529888    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e\": container with ID starting with a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e not found: ID does not exist" containerID="a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"
Oct 30 21:48:11.529957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.529917    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e"} err="failed to get container status \"a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e\": rpc error: code = NotFound desc = could not find container \"a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e\": container with ID starting with a5e2949117ea369747c9b0f9a1bee33056673d9ba8ce6ffd52e90f8d04e8ab1e not found: ID does not exist"
Oct 30 21:48:11.604063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604035    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604193    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604351    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604424    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604461    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:11.604558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604506    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604535    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604579    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604615    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604654    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket\") pod \"83daaa67-1780-4025-8921-3466c3867f59\" (UID: \"83daaa67-1780-4025-8921-3466c3867f59\") "
Oct 30 21:48:11.604965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.604935    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:11.605175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605140    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:11.605308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605224    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:11.605463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605390    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.605463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605414    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83daaa67-1780-4025-8921-3466c3867f59-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.605463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605427    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.605463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605441    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.605756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.605476    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:11.608054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.607999    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:11.608054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.608032    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:11.608763 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.608739    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:11.619063 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.619023    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg" pod "83daaa67-1780-4025-8921-3466c3867f59" (UID: "83daaa67-1780-4025-8921-3466c3867f59"). InnerVolumeSpecName "kube-api-access-rstsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:11.706770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.706658    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rstsg\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-kube-api-access-rstsg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.706770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.706697    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83daaa67-1780-4025-8921-3466c3867f59-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.706770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.706710    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.706770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.706720    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83daaa67-1780-4025-8921-3466c3867f59-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.706770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.706742    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83daaa67-1780-4025-8921-3466c3867f59-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:11.853715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.853667    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"]
Oct 30 21:48:11.862048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:11.862016    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx"]
Oct 30 21:48:12.859599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:12.853201    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:12.996136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:12.996098    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:13.346069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:13.346035    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83daaa67-1780-4025-8921-3466c3867f59" path="/var/lib/kubelet/pods/83daaa67-1780-4025-8921-3466c3867f59/volumes"
Oct 30 21:48:14.532686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:14.532647    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:16.622857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:16.621698    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:18.914051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:18.914014    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.117778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.117746    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.409567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.409532    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.505135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.505051    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"]
Oct 30 21:48:20.506132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.506069    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl" podUID="66d5093c-7ee9-4d4b-b666-f529a6bf492d" containerName="payload-pre-processing" containerID="cri-o://646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34" gracePeriod=30
Oct 30 21:48:20.610247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.610214    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.765654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.764728    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.854290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.854261    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:48:21.148953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:20.998819    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:21.148953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.039432    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume\") pod \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") "
Oct 30 21:48:21.148953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.046959    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm\" (UniqueName: \"kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm\") pod \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\" (UID: \"66d5093c-7ee9-4d4b-b666-f529a6bf492d\") "
Oct 30 21:48:21.148953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.061118    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume" pod "66d5093c-7ee9-4d4b-b666-f529a6bf492d" (UID: "66d5093c-7ee9-4d4b-b666-f529a6bf492d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:21.148953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.105149    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm" pod "66d5093c-7ee9-4d4b-b666-f529a6bf492d" (UID: "66d5093c-7ee9-4d4b-b666-f529a6bf492d"). InnerVolumeSpecName "kube-api-access-8npkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:21.156880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.155979    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8npkm\" (UniqueName: \"kubernetes.io/projected/66d5093c-7ee9-4d4b-b666-f529a6bf492d-kube-api-access-8npkm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:21.156880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.156019    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d5093c-7ee9-4d4b-b666-f529a6bf492d-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:21.181685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.181643    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:21.192231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.192069    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"]
Oct 30 21:48:21.193681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.193542    4549 cpu_manager.go:414] "containerMap: removing container" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy"
Oct 30 21:48:21.193886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.193867    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy"
Oct 30 21:48:21.194043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194030    4549 cpu_manager.go:414] "containerMap: removing container" podUID="66d5093c-7ee9-4d4b-b666-f529a6bf492d" containerName="payload-pre-processing"
Oct 30 21:48:21.194150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194135    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66d5093c-7ee9-4d4b-b666-f529a6bf492d" containerName="payload-pre-processing"
Oct 30 21:48:21.194251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194239    4549 cpu_manager.go:414] "containerMap: removing container" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy"
Oct 30 21:48:21.194340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194328    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy"
Oct 30 21:48:21.194677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194645    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy"
Oct 30 21:48:21.194844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194816    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="66d5093c-7ee9-4d4b-b666-f529a6bf492d" containerName="payload-pre-processing"
Oct 30 21:48:21.194942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.194933    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="83daaa67-1780-4025-8921-3466c3867f59" containerName="istio-proxy"
Oct 30 21:48:21.228658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.228595    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"]
Oct 30 21:48:21.228873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.228805    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.251219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.251191    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-8964ac-openshift-default-dockercfg-q8w4g\""
Oct 30 21:48:21.257508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257469    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.257700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257534    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.257700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257637    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsf4d\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.257700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257678    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.257901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257772    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.257901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257795    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.257901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257858    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.258197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257907    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.258197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.257975    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.341856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.341780    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:21.359346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359319    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.359522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359374    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.359522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359424    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.359809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359532    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.359809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359559    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.359809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359601    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsf4d\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.359809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359658    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359806    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.359999    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.360038    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.360062    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360211 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.360122    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.360240    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.360408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.360380    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.362549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.362527    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.362652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.362631    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.412529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.407684    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsf4d\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.412529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.412221    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token\") pod \"e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") " pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.521853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.521597    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:21.554794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.554762    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:21.642942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.642897    4549 generic.go:356] "Generic (PLEG): container finished" podID="66d5093c-7ee9-4d4b-b666-f529a6bf492d" containerID="646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34" exitCode=0
Oct 30 21:48:21.643108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.643039    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl" event={"ID":"66d5093c-7ee9-4d4b-b666-f529a6bf492d","Type":"ContainerDied","Data":"646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34"}
Oct 30 21:48:21.643108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.643087    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl" event={"ID":"66d5093c-7ee9-4d4b-b666-f529a6bf492d","Type":"ContainerDied","Data":"ca5b96e8f7571f3d3c84757a00981d0aea66d41ab9875290d4212ae43ab6e703"}
Oct 30 21:48:21.643225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.643113    4549 scope.go:122] "RemoveContainer" containerID="646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34"
Oct 30 21:48:21.643369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.643351    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"
Oct 30 21:48:21.653309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.653058    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:21.674268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.674215    4549 scope.go:122] "RemoveContainer" containerID="646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34"
Oct 30 21:48:21.674504 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:21.674479    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34\": container with ID starting with 646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34 not found: ID does not exist" containerID="646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34"
Oct 30 21:48:21.674591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.674516    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34"} err="failed to get container status \"646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34\": rpc error: code = NotFound desc = could not find container \"646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34\": container with ID starting with 646f8eead55e8c9a5a90fc5c017ad14c2d84228d2cfed7aee2543f7d80569c34 not found: ID does not exist"
Oct 30 21:48:21.713905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.705665    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"]
Oct 30 21:48:21.713905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.709967    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl"]
Oct 30 21:48:21.974403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.974377    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"]
Oct 30 21:48:21.975320 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:48:21.975295    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922ccaa2_5585_4c02_9305_8d2efe80d85f.slice/crio-00e58af23f8f92ab969e1ae3942436e34a38a5f10b1e71296fc9588485bbafc5 WatchSource:0}: Error finding container 00e58af23f8f92ab969e1ae3942436e34a38a5f10b1e71296fc9588485bbafc5: Status 404 returned error can't find the container with id 00e58af23f8f92ab969e1ae3942436e34a38a5f10b1e71296fc9588485bbafc5
Oct 30 21:48:21.979048 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.979007    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:21.979130 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.979102    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:21.979172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:21.979146    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:22.653105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:22.653063    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" event={"ID":"922ccaa2-5585-4c02-9305-8d2efe80d85f","Type":"ContainerStarted","Data":"4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"}
Oct 30 21:48:22.653622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:22.653158    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" event={"ID":"922ccaa2-5585-4c02-9305-8d2efe80d85f","Type":"ContainerStarted","Data":"00e58af23f8f92ab969e1ae3942436e34a38a5f10b1e71296fc9588485bbafc5"}
Oct 30 21:48:22.713134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:22.713081    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podStartSLOduration=1.713068061 podStartE2EDuration="1.713068061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:22.70941994 +0000 UTC m=+2885.483819880" watchObservedRunningTime="2026-10-30 21:48:22.713068061 +0000 UTC m=+2885.487468000"
Oct 30 21:48:23.231515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:23.231467    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:23.348009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:23.347966    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66d5093c-7ee9-4d4b-b666-f529a6bf492d" path="/var/lib/kubelet/pods/66d5093c-7ee9-4d4b-b666-f529a6bf492d/volumes"
Oct 30 21:48:23.557249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:23.557160    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:23.564923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:23.564890    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:23.660686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:23.660646    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:23.661872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:23.661843    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:48:24.955979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:24.955937    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:25.163814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:25.163766    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:25.357680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:25.357648    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:26.576031 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:26.575990    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:48:26.681807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.681772    4549 generic.go:356] "Generic (PLEG): container finished" podID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerID="c906afc1ca3ebdba8d04b12523881b6ce5c1b5cf73b21bbc15193179780c7178" exitCode=0
Oct 30 21:48:26.682008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.681874    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" event={"ID":"71c031ff-19b0-42ba-9ba3-f9049f270dbb","Type":"ContainerDied","Data":"c906afc1ca3ebdba8d04b12523881b6ce5c1b5cf73b21bbc15193179780c7178"}
Oct 30 21:48:26.682359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.682334    4549 scope.go:122] "RemoveContainer" containerID="c906afc1ca3ebdba8d04b12523881b6ce5c1b5cf73b21bbc15193179780c7178"
Oct 30 21:48:26.684365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.684336    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:26.684440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.684414    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:26.684482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.684445    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:26.731446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.731408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:26.893948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:26.893855    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:27.101153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:27.101111    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:27.691650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:27.691618    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" event={"ID":"71c031ff-19b0-42ba-9ba3-f9049f270dbb","Type":"ContainerStarted","Data":"b961d1a4b419b79d26a7ff166c047cf37aab55750ff0c31363e22af2438eac25"}
Oct 30 21:48:28.633580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:28.633536    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:48:28.633780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:28.633761    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 21:48:28.633868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:28.633821    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 21:48:28.827291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:28.827251    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:29.632907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:29.632867    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 21:48:29.633113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:29.632955    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 21:48:29.911800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:29.911708    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:30.109593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:30.109563    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:30.380466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:30.380428    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:30.633774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:30.633681    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 21:48:30.633774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:30.633735    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 21:48:30.917644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:30.917559    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:31.633775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:31.633738    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 21:48:31.633993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:31.633793    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 21:48:32.633851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:32.632822    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 21:48:32.633851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:32.632912    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 21:48:32.916627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:32.916525    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:33.633226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:33.633188    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 21:48:33.633468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:33.633254    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 21:48:34.142105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.142064    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"]
Oct 30 21:48:34.175945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.175902    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"]
Oct 30 21:48:34.330036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.330001    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:34.517179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.517036    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:34.638550 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.638520    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:48:34.639095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.639075    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:48:34.639819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.639801    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:48:34.784637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:34.784538    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:36.638481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:36.638444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:36.815143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:36.815108    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:38.452638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:38.452586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:39.143290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.143229    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz" podUID="3ba21ca0-ef35-43b0-91df-bb17138968bf" containerName="payload-processing" containerID="cri-o://1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658" gracePeriod=25
Oct 30 21:48:39.176418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.176381    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls" podUID="7c71507f-77c9-4017-8d88-9ef13130ec04" containerName="payload-pre-processing" containerID="cri-o://42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc" gracePeriod=25
Oct 30 21:48:39.455152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.455122    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:48:39.515572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.515544    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:48:39.600124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.600093    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume\") pod \"7c71507f-77c9-4017-8d88-9ef13130ec04\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") "
Oct 30 21:48:39.600378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.600363    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b\" (UniqueName: \"kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b\") pod \"3ba21ca0-ef35-43b0-91df-bb17138968bf\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") "
Oct 30 21:48:39.600464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.600397    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2\" (UniqueName: \"kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2\") pod \"7c71507f-77c9-4017-8d88-9ef13130ec04\" (UID: \"7c71507f-77c9-4017-8d88-9ef13130ec04\") "
Oct 30 21:48:39.600464 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.600454    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume\") pod \"3ba21ca0-ef35-43b0-91df-bb17138968bf\" (UID: \"3ba21ca0-ef35-43b0-91df-bb17138968bf\") "
Oct 30 21:48:39.600586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.600529    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume" pod "7c71507f-77c9-4017-8d88-9ef13130ec04" (UID: "7c71507f-77c9-4017-8d88-9ef13130ec04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:39.601207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.601154    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume" pod "3ba21ca0-ef35-43b0-91df-bb17138968bf" (UID: "3ba21ca0-ef35-43b0-91df-bb17138968bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:39.602008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.601979    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba21ca0-ef35-43b0-91df-bb17138968bf-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:39.602117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.602011    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c71507f-77c9-4017-8d88-9ef13130ec04-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:39.603120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.603097    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2" pod "7c71507f-77c9-4017-8d88-9ef13130ec04" (UID: "7c71507f-77c9-4017-8d88-9ef13130ec04"). InnerVolumeSpecName "kube-api-access-bf2v2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:39.603120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.603114    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b" pod "3ba21ca0-ef35-43b0-91df-bb17138968bf" (UID: "3ba21ca0-ef35-43b0-91df-bb17138968bf"). InnerVolumeSpecName "kube-api-access-2566b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:39.703419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.703320    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2566b\" (UniqueName: \"kubernetes.io/projected/3ba21ca0-ef35-43b0-91df-bb17138968bf-kube-api-access-2566b\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:39.703419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.703355    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf2v2\" (UniqueName: \"kubernetes.io/projected/7c71507f-77c9-4017-8d88-9ef13130ec04-kube-api-access-bf2v2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:39.782149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.782110    4549 generic.go:356] "Generic (PLEG): container finished" podID="3ba21ca0-ef35-43b0-91df-bb17138968bf" containerID="1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658" exitCode=0
Oct 30 21:48:39.782356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.782162    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz" event={"ID":"3ba21ca0-ef35-43b0-91df-bb17138968bf","Type":"ContainerDied","Data":"1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658"}
Oct 30 21:48:39.782356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.782179    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"
Oct 30 21:48:39.782356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.782222    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz" event={"ID":"3ba21ca0-ef35-43b0-91df-bb17138968bf","Type":"ContainerDied","Data":"f33b38e1062a7aac00e436742b43132203d1919d3773c452bdf2d4f70916c0bd"}
Oct 30 21:48:39.782356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.782251    4549 scope.go:122] "RemoveContainer" containerID="1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658"
Oct 30 21:48:39.783909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.783892    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"
Oct 30 21:48:39.783909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.783819    4549 generic.go:356] "Generic (PLEG): container finished" podID="7c71507f-77c9-4017-8d88-9ef13130ec04" containerID="42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc" exitCode=0
Oct 30 21:48:39.784077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.783926    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls" event={"ID":"7c71507f-77c9-4017-8d88-9ef13130ec04","Type":"ContainerDied","Data":"42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc"}
Oct 30 21:48:39.784077 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.783977    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls" event={"ID":"7c71507f-77c9-4017-8d88-9ef13130ec04","Type":"ContainerDied","Data":"dc4f8535c9917617b705ed0dd24c62759d40325f2ccd40c1318425d6631cc43a"}
Oct 30 21:48:39.793864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.793685    4549 scope.go:122] "RemoveContainer" containerID="1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658"
Oct 30 21:48:39.794159 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:39.794131    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658\": container with ID starting with 1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658 not found: ID does not exist" containerID="1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658"
Oct 30 21:48:39.794218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.794172    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658"} err="failed to get container status \"1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658\": rpc error: code = NotFound desc = could not find container \"1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658\": container with ID starting with 1e1684138cd3d4722c7fe27bc9deecfc951de6104cca153a405e200fffcf4658 not found: ID does not exist"
Oct 30 21:48:39.794218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.794196    4549 scope.go:122] "RemoveContainer" containerID="42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc"
Oct 30 21:48:39.807752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.807732    4549 scope.go:122] "RemoveContainer" containerID="42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc"
Oct 30 21:48:39.808087 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:39.808067    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc\": container with ID starting with 42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc not found: ID does not exist" containerID="42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc"
Oct 30 21:48:39.808148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.808097    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc"} err="failed to get container status \"42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc\": rpc error: code = NotFound desc = could not find container \"42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc\": container with ID starting with 42f0e5170e332df6c639e124b8e53be838ff6a948a534469e28037a62dd30efc not found: ID does not exist"
Oct 30 21:48:39.861571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.861535    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"]
Oct 30 21:48:39.869041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.869010    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls"]
Oct 30 21:48:39.917892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.917859    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"]
Oct 30 21:48:39.930880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:39.930851    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz"]
Oct 30 21:48:40.100250 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:40.100219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:41.346473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:41.346435    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ba21ca0-ef35-43b0-91df-bb17138968bf" path="/var/lib/kubelet/pods/3ba21ca0-ef35-43b0-91df-bb17138968bf/volumes"
Oct 30 21:48:41.347036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:41.347015    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c71507f-77c9-4017-8d88-9ef13130ec04" path="/var/lib/kubelet/pods/7c71507f-77c9-4017-8d88-9ef13130ec04/volumes"
Oct 30 21:48:41.374903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:41.374864    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"]
Oct 30 21:48:41.621037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:41.620915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:41.869113 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:41.869076    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:42.664927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:42.664887    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:43.367935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:43.367870    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:44.507214 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:44.507085    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"]
Oct 30 21:48:44.508640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:44.508582    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4" podUID="1a572e56-2552-49be-94ee-00233fdae91a" containerName="maas-api" containerID="cri-o://bfceac72fa027a8dbaa150f6f468fafdfd21c58ba46c5592620c04c4f5777eba" gracePeriod=30
Oct 30 21:48:44.597142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:44.597109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:44.848982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:44.848734    4549 generic.go:356] "Generic (PLEG): container finished" podID="1a572e56-2552-49be-94ee-00233fdae91a" containerID="bfceac72fa027a8dbaa150f6f468fafdfd21c58ba46c5592620c04c4f5777eba" exitCode=0
Oct 30 21:48:44.848982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:44.848869    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4" event={"ID":"1a572e56-2552-49be-94ee-00233fdae91a","Type":"ContainerDied","Data":"bfceac72fa027a8dbaa150f6f468fafdfd21c58ba46c5592620c04c4f5777eba"}
Oct 30 21:48:44.977480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:44.977452    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:48:45.084513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.084480    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:45.157845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.157739    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca\") pod \"1a572e56-2552-49be-94ee-00233fdae91a\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") "
Oct 30 21:48:45.157845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.157788    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg\" (UniqueName: \"kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg\") pod \"1a572e56-2552-49be-94ee-00233fdae91a\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") "
Oct 30 21:48:45.158059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.157937    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs\") pod \"1a572e56-2552-49be-94ee-00233fdae91a\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") "
Oct 30 21:48:45.158059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.158014    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls\") pod \"1a572e56-2552-49be-94ee-00233fdae91a\" (UID: \"1a572e56-2552-49be-94ee-00233fdae91a\") "
Oct 30 21:48:45.158216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.158193    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca" pod "1a572e56-2552-49be-94ee-00233fdae91a" (UID: "1a572e56-2552-49be-94ee-00233fdae91a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:45.158549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.158525    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a572e56-2552-49be-94ee-00233fdae91a-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:45.160163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.160130    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs" pod "1a572e56-2552-49be-94ee-00233fdae91a" (UID: "1a572e56-2552-49be-94ee-00233fdae91a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:45.160263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.160194    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls" pod "1a572e56-2552-49be-94ee-00233fdae91a" (UID: "1a572e56-2552-49be-94ee-00233fdae91a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:45.160263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.160198    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg" pod "1a572e56-2552-49be-94ee-00233fdae91a" (UID: "1a572e56-2552-49be-94ee-00233fdae91a"). InnerVolumeSpecName "kube-api-access-xcwmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:45.239329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.239289    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:45.259786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.259751    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:45.259786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.259785    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcwmg\" (UniqueName: \"kubernetes.io/projected/1a572e56-2552-49be-94ee-00233fdae91a-kube-api-access-xcwmg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:45.259786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.259795    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a572e56-2552-49be-94ee-00233fdae91a-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:45.402457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.402425    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:45.856789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.856758    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"
Oct 30 21:48:45.857267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.856794    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4" event={"ID":"1a572e56-2552-49be-94ee-00233fdae91a","Type":"ContainerDied","Data":"2c08822a1034ce600712f7ec69fc791188da9d6a5aa425e3631d9d6c4d5cd7e9"}
Oct 30 21:48:45.857267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.856879    4549 scope.go:122] "RemoveContainer" containerID="bfceac72fa027a8dbaa150f6f468fafdfd21c58ba46c5592620c04c4f5777eba"
Oct 30 21:48:45.948365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.948330    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"]
Oct 30 21:48:45.977490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:45.977458    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a4782a-5778b97c-cssb4"]
Oct 30 21:48:46.040337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:46.040300    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:46.376169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:46.376116    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerName="payload-pre-processing" containerID="cri-o://c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b" gracePeriod=25
Oct 30 21:48:47.348303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:47.348267    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a572e56-2552-49be-94ee-00233fdae91a" path="/var/lib/kubelet/pods/1a572e56-2552-49be-94ee-00233fdae91a/volumes"
Oct 30 21:48:47.510855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:47.508298    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:49.367976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:49.367942    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:50.988078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:50.988036    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:52.689639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:52.689587    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:52.918203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:52.918151    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:53.125996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:53.125955    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:53.200412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:53.200378    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"]
Oct 30 21:48:53.200629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:53.200603    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" containerID="cri-o://72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c" gracePeriod=30
Oct 30 21:48:53.366627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:53.364777    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:53.634643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:53.634600    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:48:54.132597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.132559    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"]
Oct 30 21:48:54.133581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133560    4549 cpu_manager.go:414] "containerMap: removing container" podUID="1a572e56-2552-49be-94ee-00233fdae91a" containerName="maas-api"
Oct 30 21:48:54.133661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133586    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a572e56-2552-49be-94ee-00233fdae91a" containerName="maas-api"
Oct 30 21:48:54.133661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133620    4549 cpu_manager.go:414] "containerMap: removing container" podUID="7c71507f-77c9-4017-8d88-9ef13130ec04" containerName="payload-pre-processing"
Oct 30 21:48:54.133661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133628    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c71507f-77c9-4017-8d88-9ef13130ec04" containerName="payload-pre-processing"
Oct 30 21:48:54.133661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133644    4549 cpu_manager.go:414] "containerMap: removing container" podUID="3ba21ca0-ef35-43b0-91df-bb17138968bf" containerName="payload-processing"
Oct 30 21:48:54.133661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133651    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ba21ca0-ef35-43b0-91df-bb17138968bf" containerName="payload-processing"
Oct 30 21:48:54.133902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133813    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a572e56-2552-49be-94ee-00233fdae91a" containerName="maas-api"
Oct 30 21:48:54.133902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133849    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c71507f-77c9-4017-8d88-9ef13130ec04" containerName="payload-pre-processing"
Oct 30 21:48:54.133902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.133861    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ba21ca0-ef35-43b0-91df-bb17138968bf" containerName="payload-processing"
Oct 30 21:48:54.137777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.137757    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.154270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.154247    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-889195\"/\"openshift-service-ca.crt\""
Oct 30 21:48:54.165907 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.165884    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-889195\"/\"kube-root-ca.crt\""
Oct 30 21:48:54.172727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.172696    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"]
Oct 30 21:48:54.207581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.207531    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.207768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.207747    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cpm6\" (UniqueName: \"kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.207839 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.207785    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.207933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.207913    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.207999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.207964    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.208166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.208140    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.216290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.216261    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-889195\"/\"default-dockercfg-jqrdk\""
Oct 30 21:48:54.216431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.216262    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-889195\"/\"sub-test-model-889195-kserve-self-signed-certs\""
Oct 30 21:48:54.239015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.238980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:54.310018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.309981    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cpm6\" (UniqueName: \"kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310027    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310072    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310107    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310227    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310325    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310521    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.310633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.310612    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.311111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.311089    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.312640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.312612    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.312905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.312886    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.329877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.329846    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cpm6\" (UniqueName: \"kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6\") pod \"sub-test-model-889195-kserve-cb5b88f45-6qx9g\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") " pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.432641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.432531    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:54.478236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.478198    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:54.849505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.849479    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"]
Oct 30 21:48:54.850859 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:48:54.850807    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61b35dec_44ce_4347_aab1_2d19496c87de.slice/crio-550d2c0fa4779e8e1d570d7eca2d4e0d03aaf55c00c11789516424606ade9c32 WatchSource:0}: Error finding container 550d2c0fa4779e8e1d570d7eca2d4e0d03aaf55c00c11789516424606ade9c32: Status 404 returned error can't find the container with id 550d2c0fa4779e8e1d570d7eca2d4e0d03aaf55c00c11789516424606ade9c32
Oct 30 21:48:54.936222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.936192    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g" event={"ID":"61b35dec-44ce-4347-aab1-2d19496c87de","Type":"ContainerStarted","Data":"69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c"}
Oct 30 21:48:54.936358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.936238    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g" event={"ID":"61b35dec-44ce-4347-aab1-2d19496c87de","Type":"ContainerStarted","Data":"550d2c0fa4779e8e1d570d7eca2d4e0d03aaf55c00c11789516424606ade9c32"}
Oct 30 21:48:54.978051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:54.978001    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g" podStartSLOduration=0.977984693 podStartE2EDuration="977.984693ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:54.975019882 +0000 UTC m=+2917.749419820" watchObservedRunningTime="2026-10-30 21:48:54.977984693 +0000 UTC m=+2917.752384632"
Oct 30 21:48:55.480658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:55.480627    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:48:56.219424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:56.219393    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:56.573859 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:56.573786    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:48:56.776492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:56.776449    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:48:56.776694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:56.776520    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:48:57.936148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:57.936080    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:58.503238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.503213    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:48:58.593944 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.593906    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.593969    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594075    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594108    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594149    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594264    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594297    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594368    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594414    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:58.594495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594427    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data\") pod \"b6b997d8-5e21-4c24-a638-b193aaf3d240\" (UID: \"b6b997d8-5e21-4c24-a638-b193aaf3d240\") "
Oct 30 21:48:58.594623 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594597    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:58.594847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594804    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:58.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.594969    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:58.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.595246    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.595267    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.595281    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.595294    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.595659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.595624    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:58.598977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.598949    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:48:58.599098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.599006    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "kube-api-access-2fzsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:58.599098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.599030    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:58.599098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.599034    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy" pod "b6b997d8-5e21-4c24-a638-b193aaf3d240" (UID: "b6b997d8-5e21-4c24-a638-b193aaf3d240"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:48:58.697152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.697119    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.697152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.697151    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fzsf\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-kube-api-access-2fzsf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.697152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.697161    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6b997d8-5e21-4c24-a638-b193aaf3d240-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.697416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.697171    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.697416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.697179    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b997d8-5e21-4c24-a638-b193aaf3d240-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:48:58.975920 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.975878    4549 generic.go:356] "Generic (PLEG): container finished" podID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerID="72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c" exitCode=0
Oct 30 21:48:58.976443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.975956    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"
Oct 30 21:48:58.976443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.975958    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" event={"ID":"b6b997d8-5e21-4c24-a638-b193aaf3d240","Type":"ContainerDied","Data":"72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c"}
Oct 30 21:48:58.976443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.976016    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw" event={"ID":"b6b997d8-5e21-4c24-a638-b193aaf3d240","Type":"ContainerDied","Data":"7f205e13848fe888492706b3b7242c2bb611b38f87d3ce2f31f4159fb5998c86"}
Oct 30 21:48:58.976443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.976039    4549 scope.go:122] "RemoveContainer" containerID="72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c"
Oct 30 21:48:58.987678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.987655    4549 scope.go:122] "RemoveContainer" containerID="72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c"
Oct 30 21:48:58.988025 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:48:58.988003    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c\": container with ID starting with 72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c not found: ID does not exist" containerID="72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c"
Oct 30 21:48:58.988091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:58.988034    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c"} err="failed to get container status \"72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c\": rpc error: code = NotFound desc = could not find container \"72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c\": container with ID starting with 72d06a7067c4209264dbf97e0c995320505638a8228ce40919e077c6ea5e9e5c not found: ID does not exist"
Oct 30 21:48:59.029366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:59.029329    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"]
Oct 30 21:48:59.034491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:59.034459    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw"]
Oct 30 21:48:59.346634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:59.346592    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" path="/var/lib/kubelet/pods/b6b997d8-5e21-4c24-a638-b193aaf3d240/volumes"
Oct 30 21:48:59.444634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:48:59.444602    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:01.141520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:01.141488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:03.148291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:03.148256    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:03.634388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:03.634340    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:04.246702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.246655    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"]
Oct 30 21:49:04.280090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.280053    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"]
Oct 30 21:49:04.402947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.402909    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:04.505896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.505790    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:49:04.530059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.530030    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:49:04.560694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.560653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:04.730375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:04.730343    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.029231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:05.029188    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:49:06.309389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.309352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:06.666526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.666496    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:49:06.818356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.818319    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume\") pod \"8f333559-04dc-4362-82d8-8bbdec4f743c\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") "
Oct 30 21:49:06.818547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.818369    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6\" (UniqueName: \"kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6\") pod \"8f333559-04dc-4362-82d8-8bbdec4f743c\" (UID: \"8f333559-04dc-4362-82d8-8bbdec4f743c\") "
Oct 30 21:49:06.818769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.818744    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume" pod "8f333559-04dc-4362-82d8-8bbdec4f743c" (UID: "8f333559-04dc-4362-82d8-8bbdec4f743c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:06.820723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.820700    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6" pod "8f333559-04dc-4362-82d8-8bbdec4f743c" (UID: "8f333559-04dc-4362-82d8-8bbdec4f743c"). InnerVolumeSpecName "kube-api-access-lj2q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:06.920393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.920306    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f333559-04dc-4362-82d8-8bbdec4f743c-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:06.920393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:06.920337    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj2q6\" (UniqueName: \"kubernetes.io/projected/8f333559-04dc-4362-82d8-8bbdec4f743c-kube-api-access-lj2q6\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:07.034768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.034724    4549 generic.go:356] "Generic (PLEG): container finished" podID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerID="c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b" exitCode=0
Oct 30 21:49:07.035015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.034796    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"
Oct 30 21:49:07.035015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.034862    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" event={"ID":"8f333559-04dc-4362-82d8-8bbdec4f743c","Type":"ContainerDied","Data":"c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b"}
Oct 30 21:49:07.035015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.034940    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc" event={"ID":"8f333559-04dc-4362-82d8-8bbdec4f743c","Type":"ContainerDied","Data":"f2cac8efa61dff0e0202390d23216118216a653e53f24eb12f63f83f3e59357f"}
Oct 30 21:49:07.035015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.034978    4549 scope.go:122] "RemoveContainer" containerID="c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b"
Oct 30 21:49:07.046151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.046128    4549 scope.go:122] "RemoveContainer" containerID="c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b"
Oct 30 21:49:07.046492 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:07.046471    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b\": container with ID starting with c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b not found: ID does not exist" containerID="c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b"
Oct 30 21:49:07.046555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.046502    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b"} err="failed to get container status \"c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b\": rpc error: code = NotFound desc = could not find container \"c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b\": container with ID starting with c5d03090f44753bf7c2c97e76862ecddb9e7751abc46404874bc48a1330d796b not found: ID does not exist"
Oct 30 21:49:07.112041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.112008    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"]
Oct 30 21:49:07.129860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.129812    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc"]
Oct 30 21:49:07.345872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.345818    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" path="/var/lib/kubelet/pods/8f333559-04dc-4362-82d8-8bbdec4f743c/volumes"
Oct 30 21:49:07.992078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:07.992041    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:08.142382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:08.141690    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:09.247781 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:09.247735    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerName="payload-processing" containerID="cri-o://b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1" gracePeriod=25
Oct 30 21:49:09.280902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:09.280853    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" containerName="payload-pre-processing" containerID="cri-o://cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416" gracePeriod=25
Oct 30 21:49:09.643882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:09.643824    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.358026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:10.357997    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:11.288900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:11.288859    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:12.952166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:12.952118    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:14.604628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:14.604580    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:15.631656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:15.631620    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:15.774085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:15.774051    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.009245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.009146    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"]
Oct 30 21:49:16.010542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.010511    4549 cpu_manager.go:414] "containerMap: removing container" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerName="payload-pre-processing"
Oct 30 21:49:16.010709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.010696    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerName="payload-pre-processing"
Oct 30 21:49:16.010977 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.010963    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy"
Oct 30 21:49:16.011082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.011071    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy"
Oct 30 21:49:16.011399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.011382    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6b997d8-5e21-4c24-a638-b193aaf3d240" containerName="istio-proxy"
Oct 30 21:49:16.011555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.011526    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f333559-04dc-4362-82d8-8bbdec4f743c" containerName="payload-pre-processing"
Oct 30 21:49:16.017638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.017608    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.042914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.041135    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"]
Oct 30 21:49:16.042914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.041380    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e1d376\""
Oct 30 21:49:16.042914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.042608    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e1d376-dockercfg-prvtq\""
Oct 30 21:49:16.053410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.053376    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.053645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.053584    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kcc\" (UniqueName: \"kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.155564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.155524    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.155760 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.155626    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kcc\" (UniqueName: \"kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.156442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.156420    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.184397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.184364    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kcc\" (UniqueName: \"kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc\") pod \"payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.184562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.184472    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"]
Oct 30 21:49:16.190358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.190328    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.210407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.210368    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"]
Oct 30 21:49:16.257206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.257166    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64w9w\" (UniqueName: \"kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w\") pod \"payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.257392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.257354    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.352578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.352541    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:16.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.359331    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.359533    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64w9w\" (UniqueName: \"kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w\") pod \"payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.360853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.360619    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.380136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.380102    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64w9w\" (UniqueName: \"kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w\") pod \"payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.514274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.514240    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:16.618279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.618184    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.681643 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:49:16.681595    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45822463_ab68_460b_9e1e_de6937b9835e.slice/crio-dff737301eb85595845e531f5b0045a650d4c88d37526dd554d75343d931403f WatchSource:0}: Error finding container dff737301eb85595845e531f5b0045a650d4c88d37526dd554d75343d931403f: Status 404 returned error can't find the container with id dff737301eb85595845e531f5b0045a650d4c88d37526dd554d75343d931403f
Oct 30 21:49:16.684327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.684297    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"]
Oct 30 21:49:16.830016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:16.829987    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"]
Oct 30 21:49:16.831904 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:49:16.831878    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6161306_e9d9_4740_9538_74358591c4af.slice/crio-fba4b4d54dc41742de60c3560dc825839d869bf37c355e88988ebe2fa8c8a1c1 WatchSource:0}: Error finding container fba4b4d54dc41742de60c3560dc825839d869bf37c355e88988ebe2fa8c8a1c1: Status 404 returned error can't find the container with id fba4b4d54dc41742de60c3560dc825839d869bf37c355e88988ebe2fa8c8a1c1
Oct 30 21:49:17.114714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.114676    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6" event={"ID":"c6161306-e9d9-4740-9538-74358591c4af","Type":"ContainerStarted","Data":"5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f"}
Oct 30 21:49:17.114972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.114758    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6" event={"ID":"c6161306-e9d9-4740-9538-74358591c4af","Type":"ContainerStarted","Data":"fba4b4d54dc41742de60c3560dc825839d869bf37c355e88988ebe2fa8c8a1c1"}
Oct 30 21:49:17.114972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.114793    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:17.116625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.116601    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2" event={"ID":"45822463-ab68-460b-9e1e-de6937b9835e","Type":"ContainerStarted","Data":"a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db"}
Oct 30 21:49:17.116734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.116649    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2" event={"ID":"45822463-ab68-460b-9e1e-de6937b9835e","Type":"ContainerStarted","Data":"dff737301eb85595845e531f5b0045a650d4c88d37526dd554d75343d931403f"}
Oct 30 21:49:17.116734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.116677    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:17.163198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.163081    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6" podStartSLOduration=1.163060411 podStartE2EDuration="1.163060411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:17.159007928 +0000 UTC m=+2939.933407867" watchObservedRunningTime="2026-10-30 21:49:17.163060411 +0000 UTC m=+2939.937460351"
Oct 30 21:49:17.215579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:17.215518    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2" podStartSLOduration=2.215498745 podStartE2EDuration="2.215498745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:17.211353409 +0000 UTC m=+2939.985753348" watchObservedRunningTime="2026-10-30 21:49:17.215498745 +0000 UTC m=+2939.989898689"
Oct 30 21:49:18.193600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:18.193564    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:18.288854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:18.287301    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:19.225017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:19.224976    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:19.943334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:19.943294    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:21.690910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:21.690866    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:23.803051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:23.803016    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:25.481913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:25.481878    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.329873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:26.329816    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"]
Oct 30 21:49:26.374545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:26.374507    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"]
Oct 30 21:49:26.420140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:26.420103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.623187 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:26.623058    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:49:26.681595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:26.681559    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:27.309948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:27.309913    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.125492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:28.125462    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:49:28.125997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:28.125535    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:49:28.193641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:28.193595    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:28.382846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:28.382738    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:29.225105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.225052    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:49:29.261845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.261804    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:29.582201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.582174    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:49:29.606162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.606132    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:49:29.675705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.675668    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume\") pod \"84ae85b9-870e-42b1-b170-7e72d4927525\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") "
Oct 30 21:49:29.675946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.675733    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume\") pod \"5565c23c-7d17-4aa2-9205-099b71e529d8\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") "
Oct 30 21:49:29.675946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.675775    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v\" (UniqueName: \"kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v\") pod \"5565c23c-7d17-4aa2-9205-099b71e529d8\" (UID: \"5565c23c-7d17-4aa2-9205-099b71e529d8\") "
Oct 30 21:49:29.676096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.675992    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx\" (UniqueName: \"kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx\") pod \"84ae85b9-870e-42b1-b170-7e72d4927525\" (UID: \"84ae85b9-870e-42b1-b170-7e72d4927525\") "
Oct 30 21:49:29.676196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.676140    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume" pod "84ae85b9-870e-42b1-b170-7e72d4927525" (UID: "84ae85b9-870e-42b1-b170-7e72d4927525"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:29.676336 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.676211    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume" pod "5565c23c-7d17-4aa2-9205-099b71e529d8" (UID: "5565c23c-7d17-4aa2-9205-099b71e529d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:29.676678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.676640    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ae85b9-870e-42b1-b170-7e72d4927525-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:29.676678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.676671    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5565c23c-7d17-4aa2-9205-099b71e529d8-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:29.679043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.679019    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx" pod "84ae85b9-870e-42b1-b170-7e72d4927525" (UID: "84ae85b9-870e-42b1-b170-7e72d4927525"). InnerVolumeSpecName "kube-api-access-2qvcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:29.679153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.679100    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v" pod "5565c23c-7d17-4aa2-9205-099b71e529d8" (UID: "5565c23c-7d17-4aa2-9205-099b71e529d8"). InnerVolumeSpecName "kube-api-access-7hw6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:29.777794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.777685    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hw6v\" (UniqueName: \"kubernetes.io/projected/5565c23c-7d17-4aa2-9205-099b71e529d8-kube-api-access-7hw6v\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:29.777794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:29.777721    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qvcx\" (UniqueName: \"kubernetes.io/projected/84ae85b9-870e-42b1-b170-7e72d4927525-kube-api-access-2qvcx\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:30.211945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.211904    4549 generic.go:356] "Generic (PLEG): container finished" podID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerID="b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1" exitCode=0
Oct 30 21:49:30.212168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.211982    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"
Oct 30 21:49:30.212258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.212228    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" event={"ID":"5565c23c-7d17-4aa2-9205-099b71e529d8","Type":"ContainerDied","Data":"b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1"}
Oct 30 21:49:30.212350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.212336    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr" event={"ID":"5565c23c-7d17-4aa2-9205-099b71e529d8","Type":"ContainerDied","Data":"82c33987d65696027623fa8c61c54ebfd0bd71ca819b3ccaadedeb2c452bd246"}
Oct 30 21:49:30.212412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.212371    4549 scope.go:122] "RemoveContainer" containerID="b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1"
Oct 30 21:49:30.214457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.214426    4549 generic.go:356] "Generic (PLEG): container finished" podID="84ae85b9-870e-42b1-b170-7e72d4927525" containerID="cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416" exitCode=0
Oct 30 21:49:30.214609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.214458    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" event={"ID":"84ae85b9-870e-42b1-b170-7e72d4927525","Type":"ContainerDied","Data":"cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416"}
Oct 30 21:49:30.214609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.214480    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"
Oct 30 21:49:30.214609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.214508    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf" event={"ID":"84ae85b9-870e-42b1-b170-7e72d4927525","Type":"ContainerDied","Data":"e270faafcbf714df6e9880255cb3f3e6261965229f7c99559dea4a9feb119dcc"}
Oct 30 21:49:30.225082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.225058    4549 scope.go:122] "RemoveContainer" containerID="b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1"
Oct 30 21:49:30.225423 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:30.225395    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1\": container with ID starting with b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1 not found: ID does not exist" containerID="b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1"
Oct 30 21:49:30.225793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.225434    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1"} err="failed to get container status \"b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1\": rpc error: code = NotFound desc = could not find container \"b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1\": container with ID starting with b0e61e40ce1b84c26b9e5542f54e5b7780a6e206f4d874ff35b33201b92ad3e1 not found: ID does not exist"
Oct 30 21:49:30.225793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.225456    4549 scope.go:122] "RemoveContainer" containerID="cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416"
Oct 30 21:49:30.236297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.236270    4549 scope.go:122] "RemoveContainer" containerID="cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416"
Oct 30 21:49:30.236658 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:30.236637    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416\": container with ID starting with cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416 not found: ID does not exist" containerID="cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416"
Oct 30 21:49:30.236737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.236666    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416"} err="failed to get container status \"cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416\": rpc error: code = NotFound desc = could not find container \"cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416\": container with ID starting with cc40f3c9f4183d67517a15b36c2b645cb530dc2e286b983d381381fd1ff7b416 not found: ID does not exist"
Oct 30 21:49:30.273924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.273892    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"]
Oct 30 21:49:30.285567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.285535    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf"]
Oct 30 21:49:30.322054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.322011    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"]
Oct 30 21:49:30.325519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.325491    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr"]
Oct 30 21:49:30.837616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:30.833430    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.330702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.330639    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq" podUID="39e44ccc-6631-48dd-b2b5-4576ae8eb674" containerName="payload-processing" containerID="cri-o://a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650" gracePeriod=25
Oct 30 21:49:31.349638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.349601    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" path="/var/lib/kubelet/pods/5565c23c-7d17-4aa2-9205-099b71e529d8/volumes"
Oct 30 21:49:31.350263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.350244    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" path="/var/lib/kubelet/pods/84ae85b9-870e-42b1-b170-7e72d4927525/volumes"
Oct 30 21:49:31.374999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.374934    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5" podUID="9e1ce0c3-9fa2-4bdb-9058-599d11e40578" containerName="payload-pre-processing" containerID="cri-o://9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef" gracePeriod=25
Oct 30 21:49:31.668870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.668820    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:49:31.693320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.693292    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:49:31.806338 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.806300    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume\") pod \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") "
Oct 30 21:49:31.806548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.806355    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb\" (UniqueName: \"kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb\") pod \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") "
Oct 30 21:49:31.806548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.806433    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk\" (UniqueName: \"kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk\") pod \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\" (UID: \"9e1ce0c3-9fa2-4bdb-9058-599d11e40578\") "
Oct 30 21:49:31.806548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.806519    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume\") pod \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\" (UID: \"39e44ccc-6631-48dd-b2b5-4576ae8eb674\") "
Oct 30 21:49:31.806916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.806885    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume" pod "9e1ce0c3-9fa2-4bdb-9058-599d11e40578" (UID: "9e1ce0c3-9fa2-4bdb-9058-599d11e40578"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:31.807059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.806960    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume" pod "39e44ccc-6631-48dd-b2b5-4576ae8eb674" (UID: "39e44ccc-6631-48dd-b2b5-4576ae8eb674"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:31.808857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.808816    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb" pod "39e44ccc-6631-48dd-b2b5-4576ae8eb674" (UID: "39e44ccc-6631-48dd-b2b5-4576ae8eb674"). InnerVolumeSpecName "kube-api-access-nkjpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:31.808936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.808877    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk" pod "9e1ce0c3-9fa2-4bdb-9058-599d11e40578" (UID: "9e1ce0c3-9fa2-4bdb-9058-599d11e40578"). InnerVolumeSpecName "kube-api-access-24ltk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:31.908015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.907915    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:31.908015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.907960    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkjpb\" (UniqueName: \"kubernetes.io/projected/39e44ccc-6631-48dd-b2b5-4576ae8eb674-kube-api-access-nkjpb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:31.908015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.907970    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24ltk\" (UniqueName: \"kubernetes.io/projected/9e1ce0c3-9fa2-4bdb-9058-599d11e40578-kube-api-access-24ltk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:31.908015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:31.907982    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e44ccc-6631-48dd-b2b5-4576ae8eb674-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:32.232473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.232375    4549 generic.go:356] "Generic (PLEG): container finished" podID="39e44ccc-6631-48dd-b2b5-4576ae8eb674" containerID="a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650" exitCode=0
Oct 30 21:49:32.232473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.232433    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"
Oct 30 21:49:32.232473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.232463    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq" event={"ID":"39e44ccc-6631-48dd-b2b5-4576ae8eb674","Type":"ContainerDied","Data":"a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650"}
Oct 30 21:49:32.232769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.232519    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq" event={"ID":"39e44ccc-6631-48dd-b2b5-4576ae8eb674","Type":"ContainerDied","Data":"afc25e77ca5372f05669a54ee7db7731a4985251696d1f6e17ed9bf973a0c462"}
Oct 30 21:49:32.232769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.232542    4549 scope.go:122] "RemoveContainer" containerID="a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650"
Oct 30 21:49:32.234027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.234002    4549 generic.go:356] "Generic (PLEG): container finished" podID="9e1ce0c3-9fa2-4bdb-9058-599d11e40578" containerID="9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef" exitCode=0
Oct 30 21:49:32.234161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.234036    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5" event={"ID":"9e1ce0c3-9fa2-4bdb-9058-599d11e40578","Type":"ContainerDied","Data":"9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef"}
Oct 30 21:49:32.234161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.234072    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5" event={"ID":"9e1ce0c3-9fa2-4bdb-9058-599d11e40578","Type":"ContainerDied","Data":"f27d1417cab0dfdb2060dc4b6bd0fd313c9c54063f980b922129c019e3417fa1"}
Oct 30 21:49:32.234161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.234116    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"
Oct 30 21:49:32.239016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.238981    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.245310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.245293    4549 scope.go:122] "RemoveContainer" containerID="a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650"
Oct 30 21:49:32.245668 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:32.245640    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650\": container with ID starting with a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650 not found: ID does not exist" containerID="a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650"
Oct 30 21:49:32.245738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.245682    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650"} err="failed to get container status \"a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650\": rpc error: code = NotFound desc = could not find container \"a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650\": container with ID starting with a703b408dcf62a97e9b686a7b88465c0fa240e938864984b830c4dce2b7e3650 not found: ID does not exist"
Oct 30 21:49:32.245738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.245707    4549 scope.go:122] "RemoveContainer" containerID="9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef"
Oct 30 21:49:32.255748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.255726    4549 scope.go:122] "RemoveContainer" containerID="9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef"
Oct 30 21:49:32.256121 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:32.256096    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef\": container with ID starting with 9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef not found: ID does not exist" containerID="9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef"
Oct 30 21:49:32.256187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.256130    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef"} err="failed to get container status \"9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef\": rpc error: code = NotFound desc = could not find container \"9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef\": container with ID starting with 9e04757d03331facec5fdfe8e870acf82c111cb4c75aec245f695313bd8f46ef not found: ID does not exist"
Oct 30 21:49:32.313220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.313186    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"]
Oct 30 21:49:32.340617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.340586    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5"]
Oct 30 21:49:32.382410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.382380    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"]
Oct 30 21:49:32.391506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.391475    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq"]
Oct 30 21:49:32.673339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.673299    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"]
Oct 30 21:49:32.674203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674179    4549 cpu_manager.go:414] "containerMap: removing container" podUID="39e44ccc-6631-48dd-b2b5-4576ae8eb674" containerName="payload-processing"
Oct 30 21:49:32.674203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674204    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39e44ccc-6631-48dd-b2b5-4576ae8eb674" containerName="payload-processing"
Oct 30 21:49:32.674345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674228    4549 cpu_manager.go:414] "containerMap: removing container" podUID="9e1ce0c3-9fa2-4bdb-9058-599d11e40578" containerName="payload-pre-processing"
Oct 30 21:49:32.674345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674234    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e1ce0c3-9fa2-4bdb-9058-599d11e40578" containerName="payload-pre-processing"
Oct 30 21:49:32.674345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674252    4549 cpu_manager.go:414] "containerMap: removing container" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" containerName="payload-pre-processing"
Oct 30 21:49:32.674345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674257    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" containerName="payload-pre-processing"
Oct 30 21:49:32.674345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674272    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerName="payload-processing"
Oct 30 21:49:32.674345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674278    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerName="payload-processing"
Oct 30 21:49:32.674549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674367    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e1ce0c3-9fa2-4bdb-9058-599d11e40578" containerName="payload-pre-processing"
Oct 30 21:49:32.674549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674376    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="84ae85b9-870e-42b1-b170-7e72d4927525" containerName="payload-pre-processing"
Oct 30 21:49:32.674549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674383    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5565c23c-7d17-4aa2-9205-099b71e529d8" containerName="payload-processing"
Oct 30 21:49:32.674549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.674394    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="39e44ccc-6631-48dd-b2b5-4576ae8eb674" containerName="payload-processing"
Oct 30 21:49:32.679794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.679765    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.718111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.718070    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-f9a8ed-openshift-default-dockercfg-ndwsj\""
Oct 30 21:49:32.720644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.720608    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"]
Oct 30 21:49:32.760325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.760279    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.820325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820278    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820333    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8cs5\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820529 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820502    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820570    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820615    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820770    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820866    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.820919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.820903    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.821056 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.821038    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.922860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.922748    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.922860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.922852    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.922894    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8cs5\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.923001    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.923054    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.923105    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.923208    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.923255    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.923405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.923279    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.935854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.924181    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.935854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.924850    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.935854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.932166    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.935854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.932687    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.935854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.932955    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.943865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.941972    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.943865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.943639    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.960871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.960808    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.975769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.975734    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:32.982854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:32.980655    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8cs5\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5\") pod \"e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") " pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:33.004171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.004133    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:33.346457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.346415    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39e44ccc-6631-48dd-b2b5-4576ae8eb674" path="/var/lib/kubelet/pods/39e44ccc-6631-48dd-b2b5-4576ae8eb674/volumes"
Oct 30 21:49:33.346955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.346939    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e1ce0c3-9fa2-4bdb-9058-599d11e40578" path="/var/lib/kubelet/pods/9e1ce0c3-9fa2-4bdb-9058-599d11e40578/volumes"
Oct 30 21:49:33.379472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.379437    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"]
Oct 30 21:49:33.380448 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:49:33.380423    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a182167_356a_4b2c_b874_eb0ea9057f39.slice/crio-a45deb11d5c816adeb58a5e2c03636b0853e7928a03d45f67107c61fff33020b WatchSource:0}: Error finding container a45deb11d5c816adeb58a5e2c03636b0853e7928a03d45f67107c61fff33020b: Status 404 returned error can't find the container with id a45deb11d5c816adeb58a5e2c03636b0853e7928a03d45f67107c61fff33020b
Oct 30 21:49:33.383430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.383390    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:33.383542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.383467    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:33.383542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:33.383499    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:34.251756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:34.251714    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" event={"ID":"0a182167-356a-4b2c-b874-eb0ea9057f39","Type":"ContainerStarted","Data":"77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"}
Oct 30 21:49:34.251986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:34.251777    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" event={"ID":"0a182167-356a-4b2c-b874-eb0ea9057f39","Type":"ContainerStarted","Data":"a45deb11d5c816adeb58a5e2c03636b0853e7928a03d45f67107c61fff33020b"}
Oct 30 21:49:34.294871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:34.294814    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:34.338853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:34.325474    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podStartSLOduration=2.325450667 podStartE2EDuration="2.325450667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:34.314077798 +0000 UTC m=+2957.088477753" watchObservedRunningTime="2026-10-30 21:49:34.325450667 +0000 UTC m=+2957.099850609"
Oct 30 21:49:35.005616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:35.005580    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:35.012385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:35.012350    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:35.259687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:35.259596    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:35.260613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:35.260592    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:49:36.146474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:36.142278    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.957407 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:36.957367    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:37.273779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:37.273696    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.047330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:38.047292    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.459808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:38.459772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.636333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:38.636253    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:39.848195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:39.848155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:41.498856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:41.496915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:42.704343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:42.704307    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"]
Oct 30 21:49:42.704896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:42.704597    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" podUID="deeb9b01-ff43-4ec4-ab26-eaecb69755eb" containerName="maas-api" containerID="cri-o://913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf" gracePeriod=30
Oct 30 21:49:42.805649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:42.805613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:43.065855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.065479    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:49:43.231865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.230879    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs\") pod \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") "
Oct 30 21:49:43.231865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.230962    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca\") pod \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") "
Oct 30 21:49:43.231865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.231004    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\") pod \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") "
Oct 30 21:49:43.231865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.231056    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx\" (UniqueName: \"kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx\") pod \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\" (UID: \"deeb9b01-ff43-4ec4-ab26-eaecb69755eb\") "
Oct 30 21:49:43.232789 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.232759    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca" pod "deeb9b01-ff43-4ec4-ab26-eaecb69755eb" (UID: "deeb9b01-ff43-4ec4-ab26-eaecb69755eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:43.247457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.247419    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx" pod "deeb9b01-ff43-4ec4-ab26-eaecb69755eb" (UID: "deeb9b01-ff43-4ec4-ab26-eaecb69755eb"). InnerVolumeSpecName "kube-api-access-mmpgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:43.247725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.247698    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls" pod "deeb9b01-ff43-4ec4-ab26-eaecb69755eb" (UID: "deeb9b01-ff43-4ec4-ab26-eaecb69755eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:43.254281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.254253    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs" pod "deeb9b01-ff43-4ec4-ab26-eaecb69755eb" (UID: "deeb9b01-ff43-4ec4-ab26-eaecb69755eb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:43.321008 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.320975    4549 generic.go:356] "Generic (PLEG): container finished" podID="deeb9b01-ff43-4ec4-ab26-eaecb69755eb" containerID="913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf" exitCode=0
Oct 30 21:49:43.321223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.321086    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" event={"ID":"deeb9b01-ff43-4ec4-ab26-eaecb69755eb","Type":"ContainerDied","Data":"913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf"}
Oct 30 21:49:43.321223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.321137    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g" event={"ID":"deeb9b01-ff43-4ec4-ab26-eaecb69755eb","Type":"ContainerDied","Data":"f71dfbaf354239dfaaa228c9fad5398b2484b1c6a8e08bd3f876f6104fe56a28"}
Oct 30 21:49:43.321223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.321166    4549 scope.go:122] "RemoveContainer" containerID="913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf"
Oct 30 21:49:43.321476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.321337    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"
Oct 30 21:49:43.332689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.332660    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:43.332802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.332695    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:43.332802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.332712    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:43.332802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.332725    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmpgx\" (UniqueName: \"kubernetes.io/projected/deeb9b01-ff43-4ec4-ab26-eaecb69755eb-kube-api-access-mmpgx\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:43.334271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.334246    4549 scope.go:122] "RemoveContainer" containerID="913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf"
Oct 30 21:49:43.334617 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:43.334594    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf\": container with ID starting with 913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf not found: ID does not exist" containerID="913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf"
Oct 30 21:49:43.334694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.334626    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf"} err="failed to get container status \"913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf\": rpc error: code = NotFound desc = could not find container \"913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf\": container with ID starting with 913db5f3904ac9ef4273aa7c27bd95b866b7f0c7c105e8e6a282ff354a7dc8bf not found: ID does not exist"
Oct 30 21:49:43.343497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.343469    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:43.375936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.375896    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"]
Oct 30 21:49:43.379103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.379072    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-75c17d-5dcb96cc48-ncx9g"]
Oct 30 21:49:43.515316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.515273    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:43.647658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.647576    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:43.975149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:43.975068    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:45.346217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:45.346181    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deeb9b01-ff43-4ec4-ab26-eaecb69755eb" path="/var/lib/kubelet/pods/deeb9b01-ff43-4ec4-ab26-eaecb69755eb/volumes"
Oct 30 21:49:45.852535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:45.850380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:47.448107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:47.448071    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:49.176006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:49.175974    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:50.671479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:50.671443    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:50.844620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:50.844584    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.244694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.244653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.631566 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.631528    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.775957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.775233    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"]
Oct 30 21:49:51.779862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.779064    4549 cpu_manager.go:414] "containerMap: removing container" podUID="deeb9b01-ff43-4ec4-ab26-eaecb69755eb" containerName="maas-api"
Oct 30 21:49:51.779862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.779090    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deeb9b01-ff43-4ec4-ab26-eaecb69755eb" containerName="maas-api"
Oct 30 21:49:51.779862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.779322    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="deeb9b01-ff43-4ec4-ab26-eaecb69755eb" containerName="maas-api"
Oct 30 21:49:51.791702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.785006    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.833856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.833521    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-22d912-openshift-default-dockercfg-8hvtr\""
Oct 30 21:49:51.833856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.833528    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"]
Oct 30 21:49:51.843136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.837108    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.857352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857306    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.857649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857361    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.857649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857411    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.857649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857444    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.857649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857469    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.857649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857565    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.857649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857633    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.858223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857662    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.858223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.857690    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkfch\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.970951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.960809    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.961966    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979292 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979267    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979393    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979438    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979468    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979600    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979688    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979720    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.979951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.979751    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkfch\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.980413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.980222    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.980413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.980338    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.980532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.980493    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:51.980770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.980709    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:52.004018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.988498    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:52.004018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.988725    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:52.004018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.994681    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"]
Oct 30 21:49:52.004018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:51.994935    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy" containerID="cri-o://f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419" gracePeriod=30
Oct 30 21:49:52.012115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.012006    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkfch\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:52.015657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.015594    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token\") pod \"e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") " pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:52.025206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.024971    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:52.129256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.129218    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:52.239960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.235755    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:52.492629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.492489    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:52.605598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.605567    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"]
Oct 30 21:49:52.606284 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:49:52.606251    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ced10c_7ba6_419e_ab0c_b062a870400d.slice/crio-4d9d5274044aba9859932401e4400ce40a7a0fe251db171c16c0c3680648972a WatchSource:0}: Error finding container 4d9d5274044aba9859932401e4400ce40a7a0fe251db171c16c0c3680648972a: Status 404 returned error can't find the container with id 4d9d5274044aba9859932401e4400ce40a7a0fe251db171c16c0c3680648972a
Oct 30 21:49:52.609884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.609846    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:52.610010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.609932    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:52.610010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.609965    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:52.824409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:52.824322    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:53.406510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:53.406469    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" event={"ID":"a1ced10c-7ba6-419e-ab0c-b062a870400d","Type":"ContainerStarted","Data":"ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"}
Oct 30 21:49:53.406785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:53.406528    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" event={"ID":"a1ced10c-7ba6-419e-ab0c-b062a870400d","Type":"ContainerStarted","Data":"4d9d5274044aba9859932401e4400ce40a7a0fe251db171c16c0c3680648972a"}
Oct 30 21:49:53.467334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:53.467274    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podStartSLOduration=2.467256926 podStartE2EDuration="2.467256926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:53.463976086 +0000 UTC m=+2976.238376028" watchObservedRunningTime="2026-10-30 21:49:53.467256926 +0000 UTC m=+2976.241656867"
Oct 30 21:49:54.129911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:54.129873    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:54.137779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:54.137748    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:54.413615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:54.413533    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:54.414505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:54.414485    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:49:54.673994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:54.673907    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.956404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:55.956365    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:56.118330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:56.118294    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:56.384526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:56.384486    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:56.576427 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:56.576388    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:49:57.218258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.218216    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.351672 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.351622    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.391691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.391666    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:49:57.463855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.444443    4549 generic.go:356] "Generic (PLEG): container finished" podID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerID="f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419" exitCode=0
Oct 30 21:49:57.463855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.444656    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" event={"ID":"f06ed5e1-a07c-43e7-8387-db7a68cc97a7","Type":"ContainerDied","Data":"f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419"}
Oct 30 21:49:57.463855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.444721    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8" event={"ID":"f06ed5e1-a07c-43e7-8387-db7a68cc97a7","Type":"ContainerDied","Data":"ad1133e4ebb15c9d647f1546480e2ef0317219fe95d121ad6af10b73cf0d7a5c"}
Oct 30 21:49:57.463855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.444754    4549 scope.go:122] "RemoveContainer" containerID="f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419"
Oct 30 21:49:57.463855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.444956    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.479923    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.479977    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480021    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480053    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480242    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480333    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480360    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480385    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.482173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.480484    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert\") pod \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\" (UID: \"f06ed5e1-a07c-43e7-8387-db7a68cc97a7\") "
Oct 30 21:49:57.484556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.482918    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:57.485114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.484713    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:57.485114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.484763    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:57.487443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.487132    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:57.488163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.488119    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:57.494112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.494066    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:57.496020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.495998    4549 scope.go:122] "RemoveContainer" containerID="922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"
Oct 30 21:49:57.496870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.496823    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:57.499572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.499540    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"]
Oct 30 21:49:57.501952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.501891    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:57.502360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.502334    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs" pod "f06ed5e1-a07c-43e7-8387-db7a68cc97a7" (UID: "f06ed5e1-a07c-43e7-8387-db7a68cc97a7"). InnerVolumeSpecName "kube-api-access-5klqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:57.528004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.527968    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.537463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.536982    4549 scope.go:122] "RemoveContainer" containerID="f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419"
Oct 30 21:49:57.537463 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:57.537411    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419\": container with ID starting with f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419 not found: ID does not exist" containerID="f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419"
Oct 30 21:49:57.537463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.537444    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419"} err="failed to get container status \"f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419\": rpc error: code = NotFound desc = could not find container \"f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419\": container with ID starting with f60036baee3b09c66489885fff25312349a49d0deb0bed4a91af7b3bb3160419 not found: ID does not exist"
Oct 30 21:49:57.537463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.537463    4549 scope.go:122] "RemoveContainer" containerID="922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"
Oct 30 21:49:57.537799 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:49:57.537773    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070\": container with ID starting with 922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070 not found: ID does not exist" containerID="922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"
Oct 30 21:49:57.537918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.537805    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070"} err="failed to get container status \"922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070\": rpc error: code = NotFound desc = could not find container \"922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070\": container with ID starting with 922ef5a70f9c2cbe65e13978b276bb471a9363338f3637c012b67e56bb328070 not found: ID does not exist"
Oct 30 21:49:57.582666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582630    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582660    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5klqs\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-kube-api-access-5klqs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582674    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582686    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582698    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582709    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582719    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582736    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.582967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.582747    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f06ed5e1-a07c-43e7-8387-db7a68cc97a7-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:49:57.696915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.692415    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.834562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.834526    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"]
Oct 30 21:49:57.838414 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:57.838382    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8"]
Oct 30 21:49:58.270973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.270924    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.361803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.361767    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"]
Oct 30 21:49:58.362499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.362480    4549 cpu_manager.go:414] "containerMap: removing container" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy"
Oct 30 21:49:58.362587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.362501    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy"
Oct 30 21:49:58.362587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.362514    4549 cpu_manager.go:414] "containerMap: removing container" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy"
Oct 30 21:49:58.362587 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.362520    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy"
Oct 30 21:49:58.362697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.362652    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy"
Oct 30 21:49:58.362697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.362661    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" containerName="istio-proxy"
Oct 30 21:49:58.369815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.369758    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.385999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.385958    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"]
Oct 30 21:49:58.387163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.387129    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-22d912\""
Oct 30 21:49:58.403458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.403425    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-22d912-dockercfg-zp2tl\""
Oct 30 21:49:58.408346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.408318    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"]
Oct 30 21:49:58.412872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.412847    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.439626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.439594    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"]
Oct 30 21:49:58.450459 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.450418    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.613253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.595710    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt86p\" (UniqueName: \"kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p\") pod \"payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.619914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.613802    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.620599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.620563    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jk5g\" (UniqueName: \"kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g\") pod \"payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.621364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.621330    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.723360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.723317    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.723528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.723461    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt86p\" (UniqueName: \"kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p\") pod \"payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.723568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.723526    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.723626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.723602    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jk5g\" (UniqueName: \"kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g\") pod \"payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.724144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.724118    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.724343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.724323    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.744906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.744811    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt86p\" (UniqueName: \"kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p\") pod \"payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:58.746579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.746553    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jk5g\" (UniqueName: \"kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g\") pod \"payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:58.989719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:58.989557    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:59.035514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.035481    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:49:59.317405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.317376    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"]
Oct 30 21:49:59.319030 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:49:59.319003    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c3b618_9256_4937_b523_c306daf63ddb.slice/crio-9d3e7b2ebd4fe726fe49760abc6a4cc69a24e21b1442858351cba9970ebe4b2e WatchSource:0}: Error finding container 9d3e7b2ebd4fe726fe49760abc6a4cc69a24e21b1442858351cba9970ebe4b2e: Status 404 returned error can't find the container with id 9d3e7b2ebd4fe726fe49760abc6a4cc69a24e21b1442858351cba9970ebe4b2e
Oct 30 21:49:59.346902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.346870    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f06ed5e1-a07c-43e7-8387-db7a68cc97a7" path="/var/lib/kubelet/pods/f06ed5e1-a07c-43e7-8387-db7a68cc97a7/volumes"
Oct 30 21:49:59.392778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.392752    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"]
Oct 30 21:49:59.394525 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:49:59.394492    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443f9b7e_4a89_4159_acc7_b6a22f79d839.slice/crio-24f59f9a84507f4dce741cd4c933d6c06d099d7f9ab66c79bfe4de50662b0684 WatchSource:0}: Error finding container 24f59f9a84507f4dce741cd4c933d6c06d099d7f9ab66c79bfe4de50662b0684: Status 404 returned error can't find the container with id 24f59f9a84507f4dce741cd4c933d6c06d099d7f9ab66c79bfe4de50662b0684
Oct 30 21:49:59.473812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.473777    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk" event={"ID":"443f9b7e-4a89-4159-acc7-b6a22f79d839","Type":"ContainerStarted","Data":"24f59f9a84507f4dce741cd4c933d6c06d099d7f9ab66c79bfe4de50662b0684"}
Oct 30 21:49:59.475775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.475746    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69" event={"ID":"15c3b618-9256-4937-b523-c306daf63ddb","Type":"ContainerStarted","Data":"0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a"}
Oct 30 21:49:59.475980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.475809    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:49:59.475980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.475849    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69" event={"ID":"15c3b618-9256-4937-b523-c306daf63ddb","Type":"ContainerStarted","Data":"9d3e7b2ebd4fe726fe49760abc6a4cc69a24e21b1442858351cba9970ebe4b2e"}
Oct 30 21:49:59.543849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:49:59.543777    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69" podStartSLOduration=1.54376289 podStartE2EDuration="1.54376289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:59.540540966 +0000 UTC m=+2982.314940903" watchObservedRunningTime="2026-10-30 21:49:59.54376289 +0000 UTC m=+2982.318162829"
Oct 30 21:50:00.484571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:00.484533    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk" event={"ID":"443f9b7e-4a89-4159-acc7-b6a22f79d839","Type":"ContainerStarted","Data":"a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19"}
Oct 30 21:50:00.485133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:00.484623    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:50:00.527908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:00.527845    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk" podStartSLOduration=2.527808528 podStartE2EDuration="2.527808528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:00.525708904 +0000 UTC m=+2983.300108842" watchObservedRunningTime="2026-10-30 21:50:00.527808528 +0000 UTC m=+2983.302208467"
Oct 30 21:50:00.733538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:00.733496    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:02.327462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:02.327423    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:03.347808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:03.347771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:03.478057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:03.478010    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:03.903709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:03.903677    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.149486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.149451    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.311244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.311209    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.716301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.716264    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"]
Oct 30 21:50:05.721068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.721045    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:05.755389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.755356    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e60e16\""
Oct 30 21:50:05.755630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.755486    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e60e16-dockercfg-bt4xx\""
Oct 30 21:50:05.760365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.760309    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"]
Oct 30 21:50:05.839496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.839450    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:05.839729 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.839525    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqffk\" (UniqueName: \"kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk\") pod \"payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:05.949874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.945766    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:05.949874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.945915    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqffk\" (UniqueName: \"kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk\") pod \"payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:05.949874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.947456    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:05.985911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:05.981785    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqffk\" (UniqueName: \"kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk\") pod \"payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:06.041679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.041602    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:06.392790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.392752    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:06.404382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.404340    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"]
Oct 30 21:50:06.406516 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:50:06.406470    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4111e34_86a0_4dd6_a4f7_f602e71d41eb.slice/crio-3489afc909ffd26797654c127d869889106e92c52fba5f819b76874e30c14984 WatchSource:0}: Error finding container 3489afc909ffd26797654c127d869889106e92c52fba5f819b76874e30c14984: Status 404 returned error can't find the container with id 3489afc909ffd26797654c127d869889106e92c52fba5f819b76874e30c14984
Oct 30 21:50:06.646971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.644184    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg" event={"ID":"a4111e34-86a0-4dd6-a4f7-f602e71d41eb","Type":"ContainerStarted","Data":"58cf30e142f82bb78c569f08681a84a9918e4268d7f498c9c478083969ed5c2c"}
Oct 30 21:50:06.646971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.644282    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg" event={"ID":"a4111e34-86a0-4dd6-a4f7-f602e71d41eb","Type":"ContainerStarted","Data":"3489afc909ffd26797654c127d869889106e92c52fba5f819b76874e30c14984"}
Oct 30 21:50:06.646971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.645449    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:06.695686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:06.695611    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg" podStartSLOduration=1.695590389 podStartE2EDuration="1.695590389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:06.69416496 +0000 UTC m=+2989.468564900" watchObservedRunningTime="2026-10-30 21:50:06.695590389 +0000 UTC m=+2989.469990330"
Oct 30 21:50:08.132785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:08.132739    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:09.889856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:09.889489    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:10.495024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:10.494986    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:50:10.601031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:10.600983    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:11.035667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:11.035629    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"]
Oct 30 21:50:11.043589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:11.043556    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8303f26-nldcz"]
Oct 30 21:50:11.345718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:11.345685    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2cd5025-ffd3-459b-b77f-40f7bcd3d25e" path="/var/lib/kubelet/pods/b2cd5025-ffd3-459b-b77f-40f7bcd3d25e/volumes"
Oct 30 21:50:11.498003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:11.497974    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:50:11.609646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:11.609608    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:12.527082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.526994    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main" containerID="cri-o://2b8db577d7a6e85ce58032de620a49c1b773924d5c0cf56522d857f8859748e2" gracePeriod=45
Oct 30 21:50:12.723744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.723706    4549 generic.go:356] "Generic (PLEG): container finished" podID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerID="2b8db577d7a6e85ce58032de620a49c1b773924d5c0cf56522d857f8859748e2" exitCode=0
Oct 30 21:50:12.723962 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.723785    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" event={"ID":"5c4e39e9-43c4-42d5-bf91-e0b660d634c7","Type":"ContainerDied","Data":"2b8db577d7a6e85ce58032de620a49c1b773924d5c0cf56522d857f8859748e2"}
Oct 30 21:50:12.806215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.806175    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:50:12.849424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849388    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home\") pod \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") "
Oct 30 21:50:12.849607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849472    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir\") pod \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") "
Oct 30 21:50:12.849607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849507    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm\") pod \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") "
Oct 30 21:50:12.849607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849547    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache\") pod \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") "
Oct 30 21:50:12.849607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849584    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs\") pod \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") "
Oct 30 21:50:12.849895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849707    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home" pod "5c4e39e9-43c4-42d5-bf91-e0b660d634c7" (UID: "5c4e39e9-43c4-42d5-bf91-e0b660d634c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:12.849895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.849712    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9\" (UniqueName: \"kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9\") pod \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\" (UID: \"5c4e39e9-43c4-42d5-bf91-e0b660d634c7\") "
Oct 30 21:50:12.851019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.850562    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache" pod "5c4e39e9-43c4-42d5-bf91-e0b660d634c7" (UID: "5c4e39e9-43c4-42d5-bf91-e0b660d634c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:12.851019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.850617    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir" pod "5c4e39e9-43c4-42d5-bf91-e0b660d634c7" (UID: "5c4e39e9-43c4-42d5-bf91-e0b660d634c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:12.851019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.850691    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:50:12.855112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.854014    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm" pod "5c4e39e9-43c4-42d5-bf91-e0b660d634c7" (UID: "5c4e39e9-43c4-42d5-bf91-e0b660d634c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:12.855605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.855573    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs" pod "5c4e39e9-43c4-42d5-bf91-e0b660d634c7" (UID: "5c4e39e9-43c4-42d5-bf91-e0b660d634c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:12.855973 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.855936    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9" pod "5c4e39e9-43c4-42d5-bf91-e0b660d634c7" (UID: "5c4e39e9-43c4-42d5-bf91-e0b660d634c7"). InnerVolumeSpecName "kube-api-access-rvlv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:12.954440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.954283    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:50:12.954440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.954322    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:50:12.954440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.954335    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:50:12.954440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.954348    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:50:12.954440 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:12.954362    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvlv9\" (UniqueName: \"kubernetes.io/projected/5c4e39e9-43c4-42d5-bf91-e0b660d634c7-kube-api-access-rvlv9\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:50:13.125398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:13.125359    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.733989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:13.733908    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"
Oct 30 21:50:13.733989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:13.733940    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb" event={"ID":"5c4e39e9-43c4-42d5-bf91-e0b660d634c7","Type":"ContainerDied","Data":"1740d49fc9dd54d9b35e89d3b080b59b07323be7c39ad0081c905c2d3f8658dd"}
Oct 30 21:50:13.734569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:13.734001    4549 scope.go:122] "RemoveContainer" containerID="2b8db577d7a6e85ce58032de620a49c1b773924d5c0cf56522d857f8859748e2"
Oct 30 21:50:13.786143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:13.786109    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"]
Oct 30 21:50:13.798198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:13.798157    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-2cda44/distinct-w2-2cda44-kserve-68468959c9-2tjhb"]
Oct 30 21:50:14.725887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:14.725849    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:15.350652 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:15.350611    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" path="/var/lib/kubelet/pods/5c4e39e9-43c4-42d5-bf91-e0b660d634c7/volumes"
Oct 30 21:50:16.317738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.317705    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:16.801606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.801520    4549 generic.go:356] "Generic (PLEG): container finished" podID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerID="4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e" exitCode=0
Oct 30 21:50:16.802024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.801596    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" event={"ID":"922ccaa2-5585-4c02-9305-8d2efe80d85f","Type":"ContainerDied","Data":"4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"}
Oct 30 21:50:16.802184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.802165    4549 scope.go:122] "RemoveContainer" containerID="4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"
Oct 30 21:50:16.803997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.803968    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:50:16.804071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.804040    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:50:16.804114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:16.804071    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:50:17.798155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:17.798119    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:17.810799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:17.810769    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" event={"ID":"922ccaa2-5585-4c02-9305-8d2efe80d85f","Type":"ContainerStarted","Data":"46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033"}
Oct 30 21:50:17.949291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:17.949254    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.084180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.084122    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.557397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.557290    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:50:18.557596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.557519    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 21:50:18.557596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.557575    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 21:50:18.678855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.675750    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:50:18.819947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.807096    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:50:18.823252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:18.823214    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:50:19.391858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:19.390980    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:19.557230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:19.557197    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 21:50:19.557430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:19.557260    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 21:50:20.557476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:20.557392    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 21:50:20.557476 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:20.557444    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 21:50:21.150446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:21.150413    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:21.557518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:21.557427    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 21:50:21.557518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:21.557482    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 21:50:22.557629 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:22.557595    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 21:50:22.558107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:22.557650    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 21:50:22.617383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:22.617348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:23.557417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:23.557377    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 21:50:23.557614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:23.557437    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 21:50:23.660627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:23.660595    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:50:24.183722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:24.183683    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.562766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:24.562683    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:50:24.563957 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:24.563936    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:50:25.808485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:25.808444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:26.110844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:26.110805    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:26.575512 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:50:26.575479    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:50:27.776990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:27.776953    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.425256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:29.425148    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:30.972357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:30.972322    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:31.566820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:31.566787    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:32.552855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:32.552801    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.811909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:33.807221    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.934282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:33.934242    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.268988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.268951    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"]
Oct 30 21:50:34.269733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.269715    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main"
Oct 30 21:50:34.269799 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.269735    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main"
Oct 30 21:50:34.269926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.269909    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c4e39e9-43c4-42d5-bf91-e0b660d634c7" containerName="main"
Oct 30 21:50:34.275373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.275345    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.295638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.295591    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"]
Oct 30 21:50:34.295889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.295754    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8964ac\""
Oct 30 21:50:34.302481 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.302450    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8964ac-dockercfg-8d2bg\""
Oct 30 21:50:34.406560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.406515    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.407043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.407014    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbrmt\" (UniqueName: \"kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt\") pod \"payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.463643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.463597    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.509181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.509116    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.509400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.509256    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbrmt\" (UniqueName: \"kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt\") pod \"payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.510180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.510155    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.530488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.530409    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbrmt\" (UniqueName: \"kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt\") pod \"payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.590349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.590307    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:34.680183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.680142    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.951284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.951241    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"]
Oct 30 21:50:34.955363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:34.955328    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.955513 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:50:34.955464    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e762a22_f651_4111_b54a_40f89f8afdad.slice/crio-70482c8206fee364f936b77c25fd613cb018ecb431503a37bdd5d5e46a72d0e3 WatchSource:0}: Error finding container 70482c8206fee364f936b77c25fd613cb018ecb431503a37bdd5d5e46a72d0e3: Status 404 returned error can't find the container with id 70482c8206fee364f936b77c25fd613cb018ecb431503a37bdd5d5e46a72d0e3
Oct 30 21:50:35.952775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:35.952730    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2" event={"ID":"6e762a22-f651-4111-b54a-40f89f8afdad","Type":"ContainerStarted","Data":"a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa"}
Oct 30 21:50:35.953367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:35.952816    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2" event={"ID":"6e762a22-f651-4111-b54a-40f89f8afdad","Type":"ContainerStarted","Data":"70482c8206fee364f936b77c25fd613cb018ecb431503a37bdd5d5e46a72d0e3"}
Oct 30 21:50:35.953367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:35.952971    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:35.996639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:35.996578    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2" podStartSLOduration=1.9965603060000001 podStartE2EDuration="1.996560306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:35.992758012 +0000 UTC m=+3018.767157956" watchObservedRunningTime="2026-10-30 21:50:35.996560306 +0000 UTC m=+3018.770960244"
Oct 30 21:50:36.479424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:36.479367    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:36.603884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:36.603510    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:36.744438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:36.744343    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.001491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:38.001454    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.510989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:39.510955    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:41.043603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:41.043551    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:42.500453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:42.500419    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:42.921273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:42.921240    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.308255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:44.308210    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:45.732677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:45.732639    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:46.962184 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:46.962141    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:50:47.658053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:47.658013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:49.124810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:49.124776    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:49.936057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:49.936020    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.355154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:50.355121    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.672430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:50.672288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.828036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:50.828000    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:52.390232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:52.390195    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:54.013817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:54.013778    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:54.194552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:54.194518    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.625710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:55.625667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.778051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:55.778013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.946906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:55.939066    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:56.575283 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:50:56.575245    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:50:57.615281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:57.615244    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:58.184312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:58.184274    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:58.337744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:58.337699    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:59.343660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:50:59.343617    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:01.061548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:01.061513    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:02.646892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:02.646821    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:03.392234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:03.391398    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:03.566083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:03.566042    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:03.729109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:03.729013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:03.864860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:03.864627    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:04.210448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:04.210406    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:05.786935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:05.786884    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:07.628883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:07.628780    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:09.022291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:09.022194    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.264937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.264902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.433737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.433696    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.454993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.454960    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.207:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:51:10.455178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.455027    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.207:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:51:10.553004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.552901    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"]
Oct 30 21:51:10.554118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.554082    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy" containerID="cri-o://73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e" gracePeriod=30
Oct 30 21:51:10.619879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.619846    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.800890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:10.796447    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:11.123216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.123178    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:11.362230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.362155    4549 generic.go:356] "Generic (PLEG): container finished" podID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerID="77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6" exitCode=0
Oct 30 21:51:11.362658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.362224    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" event={"ID":"0a182167-356a-4b2c-b874-eb0ea9057f39","Type":"ContainerDied","Data":"77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"}
Oct 30 21:51:11.362719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.362690    4549 scope.go:122] "RemoveContainer" containerID="77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"
Oct 30 21:51:11.364035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.364010    4549 generic.go:356] "Generic (PLEG): container finished" podID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerID="ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b" exitCode=0
Oct 30 21:51:11.364165 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.364085    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" event={"ID":"a1ced10c-7ba6-419e-ab0c-b062a870400d","Type":"ContainerDied","Data":"ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"}
Oct 30 21:51:11.364525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.364507    4549 scope.go:122] "RemoveContainer" containerID="ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"
Oct 30 21:51:11.365042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.365005    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:51:11.365155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.365080    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:51:11.365155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.365113    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:51:11.366297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.366269    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:51:11.366362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.366335    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:51:11.366400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:11.366370    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:51:12.377917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:12.377359    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" event={"ID":"0a182167-356a-4b2c-b874-eb0ea9057f39","Type":"ContainerStarted","Data":"e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4"}
Oct 30 21:51:12.397164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:12.397116    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" event={"ID":"a1ced10c-7ba6-419e-ab0c-b062a870400d","Type":"ContainerStarted","Data":"cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a"}
Oct 30 21:51:12.899854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:12.896491    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:13.006854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:13.006011    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:51:13.006854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:13.006590    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 30 21:51:13.006854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:13.006657    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 30 21:51:13.129884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:13.129814    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:51:13.130083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:13.129853    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused" start-of-body=
Oct 30 21:51:13.130083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:13.129958    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused"
Oct 30 21:51:14.005308 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:14.005268    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 30 21:51:14.010027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:14.009984    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 30 21:51:14.130238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:14.130203    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused" start-of-body=
Oct 30 21:51:14.130435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:14.130264    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused"
Oct 30 21:51:14.626324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:14.620004    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.006527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:15.006415    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 30 21:51:15.007223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:15.007173    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 30 21:51:15.130143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:15.130105    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused" start-of-body=
Oct 30 21:51:15.130339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:15.130157    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused"
Oct 30 21:51:16.006893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.006317    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 30 21:51:16.006893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.006387    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 30 21:51:16.064495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.064465    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:51:16.130556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.130510    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused" start-of-body=
Oct 30 21:51:16.130774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.130568    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused"
Oct 30 21:51:16.167273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167238    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167311    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167384    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167408    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167461    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167508    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167560    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167595    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.167716 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.167634    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert\") pod \"6b363bb0-0398-4bd5-84f0-5949a491cd35\" (UID: \"6b363bb0-0398-4bd5-84f0-5949a491cd35\") "
Oct 30 21:51:16.168105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.168070    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:16.168270 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.168212    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:16.168342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.168317    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:16.168402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.168339    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:16.168473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.168419    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:16.210228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.210049    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:16.210228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.210162    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "kube-api-access-77tj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:16.211156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.211080    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:16.218052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.217975    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy" pod "6b363bb0-0398-4bd5-84f0-5949a491cd35" (UID: "6b363bb0-0398-4bd5-84f0-5949a491cd35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.269963    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270006    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270020    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270035    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270049    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270061    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270073    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b363bb0-0398-4bd5-84f0-5949a491cd35-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270088    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b363bb0-0398-4bd5-84f0-5949a491cd35-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.276978 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.270100    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77tj9\" (UniqueName: \"kubernetes.io/projected/6b363bb0-0398-4bd5-84f0-5949a491cd35-kube-api-access-77tj9\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:16.394852 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.394393    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:16.484858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.481639    4549 generic.go:356] "Generic (PLEG): container finished" podID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerID="73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e" exitCode=0
Oct 30 21:51:16.484858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.481884    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" event={"ID":"6b363bb0-0398-4bd5-84f0-5949a491cd35","Type":"ContainerDied","Data":"73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e"}
Oct 30 21:51:16.484858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.481963    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v" event={"ID":"6b363bb0-0398-4bd5-84f0-5949a491cd35","Type":"ContainerDied","Data":"86de8d2ab9450b5ad8a994ae0b0caf76d5fdc0130fc4588372c2cca55814c381"}
Oct 30 21:51:16.484858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.481996    4549 scope.go:122] "RemoveContainer" containerID="73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e"
Oct 30 21:51:16.484858 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.482191    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"
Oct 30 21:51:16.534504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.534473    4549 scope.go:122] "RemoveContainer" containerID="73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e"
Oct 30 21:51:16.535009 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:51:16.534986    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e\": container with ID starting with 73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e not found: ID does not exist" containerID="73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e"
Oct 30 21:51:16.535124 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.535024    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e"} err="failed to get container status \"73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e\": rpc error: code = NotFound desc = could not find container \"73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e\": container with ID starting with 73e8285148618c4855509c0ba5e3793a957a685856ef88beb65348e5b493bb5e not found: ID does not exist"
Oct 30 21:51:16.553906 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.553869    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"]
Oct 30 21:51:16.562575 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:16.562543    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v"]
Oct 30 21:51:17.014857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:17.014302    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 30 21:51:17.014857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:17.014367    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 30 21:51:17.131112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:17.131075    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused" start-of-body=
Oct 30 21:51:17.131301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:17.131152    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.207:15021/healthz/ready\": dial tcp 10.133.0.207:15021: connect: connection refused"
Oct 30 21:51:17.347608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:17.347569    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" path="/var/lib/kubelet/pods/6b363bb0-0398-4bd5-84f0-5949a491cd35/volumes"
Oct 30 21:51:18.005230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:18.005190    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 30 21:51:18.005446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:18.005265    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 30 21:51:18.134743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:18.134699    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:51:18.135317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:18.135146    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:51:18.136038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:18.136016    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:51:18.462320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:18.462288    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:19.010091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:19.010059    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:51:19.010480 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:19.010458    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:51:19.011271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:19.011250    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:51:19.883618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:19.883583    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:20.141422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:20.141338    4549 scope.go:122] "RemoveContainer" containerID="3a0a3ffc8979c8e2390ce0f45e846131787c4028fa4e5f8830c93a1d831c86b9"
Oct 30 21:51:21.389533 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:21.389492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:22.737819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:22.737777    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"]
Oct 30 21:51:22.773071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:22.773031    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"]
Oct 30 21:51:22.898813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:22.898772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:23.031313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:23.031219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:23.278018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:23.277982    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:24.799520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:24.799478    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:26.575483 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:51:26.575439    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:51:26.637002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:26.636964    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:27.738234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:27.738188    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69" podUID="15c3b618-9256-4937-b523-c306daf63ddb" containerName="payload-processing" containerID="cri-o://0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a" gracePeriod=25
Oct 30 21:51:27.773929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:27.773883    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk" podUID="443f9b7e-4a89-4159-acc7-b6a22f79d839" containerName="payload-pre-processing" containerID="cri-o://a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19" gracePeriod=25
Oct 30 21:51:28.066372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.066341    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:51:28.089201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.089169    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:51:28.107030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.106995    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume\") pod \"15c3b618-9256-4937-b523-c306daf63ddb\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") "
Oct 30 21:51:28.107216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.107120    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g\" (UniqueName: \"kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g\") pod \"15c3b618-9256-4937-b523-c306daf63ddb\" (UID: \"15c3b618-9256-4937-b523-c306daf63ddb\") "
Oct 30 21:51:28.107458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.107435    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume" pod "15c3b618-9256-4937-b523-c306daf63ddb" (UID: "15c3b618-9256-4937-b523-c306daf63ddb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:28.107780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.107720    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c3b618-9256-4937-b523-c306daf63ddb-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:28.110084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.110056    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g" pod "15c3b618-9256-4937-b523-c306daf63ddb" (UID: "15c3b618-9256-4937-b523-c306daf63ddb"). InnerVolumeSpecName "kube-api-access-2jk5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:28.115187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.115157    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:28.208371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.208330    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume\") pod \"443f9b7e-4a89-4159-acc7-b6a22f79d839\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") "
Oct 30 21:51:28.208582 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.208396    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p\" (UniqueName: \"kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p\") pod \"443f9b7e-4a89-4159-acc7-b6a22f79d839\" (UID: \"443f9b7e-4a89-4159-acc7-b6a22f79d839\") "
Oct 30 21:51:28.208766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.208733    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume" pod "443f9b7e-4a89-4159-acc7-b6a22f79d839" (UID: "443f9b7e-4a89-4159-acc7-b6a22f79d839"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:28.209044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.208959    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/443f9b7e-4a89-4159-acc7-b6a22f79d839-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:28.209044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.208984    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jk5g\" (UniqueName: \"kubernetes.io/projected/15c3b618-9256-4937-b523-c306daf63ddb-kube-api-access-2jk5g\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:28.211492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.211466    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p" pod "443f9b7e-4a89-4159-acc7-b6a22f79d839" (UID: "443f9b7e-4a89-4159-acc7-b6a22f79d839"). InnerVolumeSpecName "kube-api-access-rt86p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:28.310274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.310190    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt86p\" (UniqueName: \"kubernetes.io/projected/443f9b7e-4a89-4159-acc7-b6a22f79d839-kube-api-access-rt86p\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:51:28.603601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.603560    4549 generic.go:356] "Generic (PLEG): container finished" podID="15c3b618-9256-4937-b523-c306daf63ddb" containerID="0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a" exitCode=0
Oct 30 21:51:28.603793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.603629    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"
Oct 30 21:51:28.603793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.603640    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69" event={"ID":"15c3b618-9256-4937-b523-c306daf63ddb","Type":"ContainerDied","Data":"0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a"}
Oct 30 21:51:28.603793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.603705    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69" event={"ID":"15c3b618-9256-4937-b523-c306daf63ddb","Type":"ContainerDied","Data":"9d3e7b2ebd4fe726fe49760abc6a4cc69a24e21b1442858351cba9970ebe4b2e"}
Oct 30 21:51:28.603793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.603729    4549 scope.go:122] "RemoveContainer" containerID="0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a"
Oct 30 21:51:28.605397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.605373    4549 generic.go:356] "Generic (PLEG): container finished" podID="443f9b7e-4a89-4159-acc7-b6a22f79d839" containerID="a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19" exitCode=0
Oct 30 21:51:28.605524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.605447    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"
Oct 30 21:51:28.605524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.605471    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk" event={"ID":"443f9b7e-4a89-4159-acc7-b6a22f79d839","Type":"ContainerDied","Data":"a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19"}
Oct 30 21:51:28.605524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.605521    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk" event={"ID":"443f9b7e-4a89-4159-acc7-b6a22f79d839","Type":"ContainerDied","Data":"24f59f9a84507f4dce741cd4c933d6c06d099d7f9ab66c79bfe4de50662b0684"}
Oct 30 21:51:28.622127 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.622102    4549 scope.go:122] "RemoveContainer" containerID="0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a"
Oct 30 21:51:28.622432 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:51:28.622415    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a\": container with ID starting with 0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a not found: ID does not exist" containerID="0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a"
Oct 30 21:51:28.622502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.622442    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a"} err="failed to get container status \"0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a\": rpc error: code = NotFound desc = could not find container \"0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a\": container with ID starting with 0c2dcf903dae5930f15ac3989833898c7754b7a93e16f50ef78016eac429712a not found: ID does not exist"
Oct 30 21:51:28.622502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.622459    4549 scope.go:122] "RemoveContainer" containerID="a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19"
Oct 30 21:51:28.632881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.632819    4549 scope.go:122] "RemoveContainer" containerID="a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19"
Oct 30 21:51:28.633279 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:51:28.633255    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19\": container with ID starting with a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19 not found: ID does not exist" containerID="a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19"
Oct 30 21:51:28.633358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.633284    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19"} err="failed to get container status \"a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19\": rpc error: code = NotFound desc = could not find container \"a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19\": container with ID starting with a44501d1f090c8c69c059e843e15ab4a6bb46bd7c33ae1c2f1c7c28082825b19 not found: ID does not exist"
Oct 30 21:51:28.655259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.655224    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"]
Oct 30 21:51:28.659595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.659562    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk"]
Oct 30 21:51:28.692871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.692813    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"]
Oct 30 21:51:28.696189 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:28.696161    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69"]
Oct 30 21:51:29.346899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:29.346858    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15c3b618-9256-4937-b523-c306daf63ddb" path="/var/lib/kubelet/pods/15c3b618-9256-4937-b523-c306daf63ddb/volumes"
Oct 30 21:51:29.347498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:29.347481    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="443f9b7e-4a89-4159-acc7-b6a22f79d839" path="/var/lib/kubelet/pods/443f9b7e-4a89-4159-acc7-b6a22f79d839/volumes"
Oct 30 21:51:29.547689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:29.547654    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:31.406444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:31.406396    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:32.822145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:32.822099    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:34.287783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:34.287749    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:35.686715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:35.686676    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:37.184358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:37.184317    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:37.300339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:37.300300    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:38.788689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:38.788654    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:40.351854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:40.350930    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:41.826868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:41.826793    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:43.382432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:43.382398    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:44.480898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.480857    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:44.622663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.622624    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:44.868601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.868556    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"]
Oct 30 21:51:44.869823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.869794    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy"
Oct 30 21:51:44.869992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.869978    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy"
Oct 30 21:51:44.870094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870083    4549 cpu_manager.go:414] "containerMap: removing container" podUID="15c3b618-9256-4937-b523-c306daf63ddb" containerName="payload-processing"
Oct 30 21:51:44.870164 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870155    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15c3b618-9256-4937-b523-c306daf63ddb" containerName="payload-processing"
Oct 30 21:51:44.870275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870265    4549 cpu_manager.go:414] "containerMap: removing container" podUID="443f9b7e-4a89-4159-acc7-b6a22f79d839" containerName="payload-pre-processing"
Oct 30 21:51:44.870340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870331    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="443f9b7e-4a89-4159-acc7-b6a22f79d839" containerName="payload-pre-processing"
Oct 30 21:51:44.870593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870581    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b363bb0-0398-4bd5-84f0-5949a491cd35" containerName="istio-proxy"
Oct 30 21:51:44.870676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870667    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="15c3b618-9256-4937-b523-c306daf63ddb" containerName="payload-processing"
Oct 30 21:51:44.870766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.870757    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="443f9b7e-4a89-4159-acc7-b6a22f79d839" containerName="payload-pre-processing"
Oct 30 21:51:44.887004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.886975    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:44.887520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.887494    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"]
Oct 30 21:51:44.919853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.919059    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f9a8ed-dockercfg-d966r\""
Oct 30 21:51:44.919853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.919178    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f9a8ed\""
Oct 30 21:51:44.984058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.984006    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwrzx\" (UniqueName: \"kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx\") pod \"payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:44.984264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:44.984078    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.052265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.052219    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"]
Oct 30 21:51:45.058161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.058130    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.076761 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.076729    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"]
Oct 30 21:51:45.102750 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.102697    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwrzx\" (UniqueName: \"kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx\") pod \"payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.102994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.102774    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.102994 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.102927    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.103143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.103027    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpz42\" (UniqueName: \"kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42\") pod \"payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.104347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.104312    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.123704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.123621    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwrzx\" (UniqueName: \"kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx\") pod \"payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.204905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.204862    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.205116 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.204981    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz42\" (UniqueName: \"kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42\") pod \"payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.205630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.205604    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.223342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.223301    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.230153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.230127    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpz42\" (UniqueName: \"kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42\") pod \"payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.379699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.379603    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:45.461897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.461018    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:45.562088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.562052    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"]
Oct 30 21:51:45.563735 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:51:45.563703    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1c02ef_de31_43d2_a814_b359ea86f26e.slice/crio-5d83e11d23911a969710461b1a3e0dd95ea6fcfcdca0d2721a668554b984b88f WatchSource:0}: Error finding container 5d83e11d23911a969710461b1a3e0dd95ea6fcfcdca0d2721a668554b984b88f: Status 404 returned error can't find the container with id 5d83e11d23911a969710461b1a3e0dd95ea6fcfcdca0d2721a668554b984b88f
Oct 30 21:51:45.694594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.694564    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"]
Oct 30 21:51:45.695009 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:51:45.694985    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda961251f_78ad_400f_8e1e_f57158b2dbfb.slice/crio-ab05467cbc3f22c42a0f40f6b2cb3a6066df2b1fb3059d3193db4a438d035c91 WatchSource:0}: Error finding container ab05467cbc3f22c42a0f40f6b2cb3a6066df2b1fb3059d3193db4a438d035c91: Status 404 returned error can't find the container with id ab05467cbc3f22c42a0f40f6b2cb3a6066df2b1fb3059d3193db4a438d035c91
Oct 30 21:51:45.757807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.757775    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2" event={"ID":"ea1c02ef-de31-43d2-a814-b359ea86f26e","Type":"ContainerStarted","Data":"4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4"}
Oct 30 21:51:45.758043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.757878    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2" event={"ID":"ea1c02ef-de31-43d2-a814-b359ea86f26e","Type":"ContainerStarted","Data":"5d83e11d23911a969710461b1a3e0dd95ea6fcfcdca0d2721a668554b984b88f"}
Oct 30 21:51:45.758118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.758065    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:45.760016 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.759981    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs" event={"ID":"a961251f-78ad-400f-8e1e-f57158b2dbfb","Type":"ContainerStarted","Data":"ab05467cbc3f22c42a0f40f6b2cb3a6066df2b1fb3059d3193db4a438d035c91"}
Oct 30 21:51:45.801608 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:45.801550    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2" podStartSLOduration=1.801536579 podStartE2EDuration="1.801536579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:45.797702576 +0000 UTC m=+3088.572102514" watchObservedRunningTime="2026-10-30 21:51:45.801536579 +0000 UTC m=+3088.575936517"
Oct 30 21:51:46.772356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:46.772301    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs" event={"ID":"a961251f-78ad-400f-8e1e-f57158b2dbfb","Type":"ContainerStarted","Data":"e5ff07f43d824e964b08f938dc60774387304c4866a69e1cd649e1750ab765ee"}
Oct 30 21:51:46.772786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:46.772370    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:51:46.816808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:46.816737    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs" podStartSLOduration=1.816714623 podStartE2EDuration="1.816714623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:46.812310471 +0000 UTC m=+3089.586710413" watchObservedRunningTime="2026-10-30 21:51:46.816714623 +0000 UTC m=+3089.591114562"
Oct 30 21:51:47.125854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:47.123453    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:48.379916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:48.375971    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:48.840570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:48.840531    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:50.335610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:50.335573    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:51.854302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:51.854264    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:53.722188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:53.722144    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:55.098493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:55.098458    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:56.472264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:56.472230    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:56.575159 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:51:56.575120    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:51:56.774069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:56.773973    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:51:57.399020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:57.398979    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.083187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:58.083156    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.216367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:58.216336    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.574933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:58.574898    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.766914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:58.766879    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.797356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:51:58.789158    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:52:00.056930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:00.056882    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.608230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:01.608196    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.753027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:01.752990    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.956398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:01.956303    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:02.920663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:02.920622    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.120882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.117318    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.483796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.483763    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.512701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.512663    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"]
Oct 30 21:52:03.517188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.517168    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.588853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.585260    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-22d912\""
Oct 30 21:52:03.588853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.585377    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-22d912-dockercfg-ppptg\""
Oct 30 21:52:03.651410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.651374    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"]
Oct 30 21:52:03.678937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.678898    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkzfk\" (UniqueName: \"kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk\") pod \"payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.692132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.692085    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.694922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.681584    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"]
Oct 30 21:52:03.695611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.682400    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.698530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.698498    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"]
Oct 30 21:52:03.698678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.698624    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:03.833911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.827934    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:03.833911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.828004    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl\") pod \"payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:03.833911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.828137    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkzfk\" (UniqueName: \"kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk\") pod \"payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.833911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.828178    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.833911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.829170    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.850739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.850692    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkzfk\" (UniqueName: \"kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk\") pod \"payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:03.929458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.929423    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:03.930038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.929485    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl\") pod \"payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:03.930556 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.930535    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:03.948624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:03.948593    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl\") pod \"payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:04.018810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.018773    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:04.133888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.133812    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:04.336854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.336768    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"]
Oct 30 21:52:04.337786 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:52:04.337752    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc8fe6a_8b7f_4b78_ab0e_05d7d6b53fda.slice/crio-d2740c6f27649f70839d249d85ac1b2ec335b03ec7568733aa219235af247fae WatchSource:0}: Error finding container d2740c6f27649f70839d249d85ac1b2ec335b03ec7568733aa219235af247fae: Status 404 returned error can't find the container with id d2740c6f27649f70839d249d85ac1b2ec335b03ec7568733aa219235af247fae
Oct 30 21:52:04.454041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.454010    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"]
Oct 30 21:52:04.455044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.455014    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:04.455977 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:52:04.455950    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fb0f05_db53_49e3_8d70_9aa2df134292.slice/crio-dd6b15e147e1605a70de5e4c75ac2b2aa47d62af13fed8c4d7c6e3f938e2d932 WatchSource:0}: Error finding container dd6b15e147e1605a70de5e4c75ac2b2aa47d62af13fed8c4d7c6e3f938e2d932: Status 404 returned error can't find the container with id dd6b15e147e1605a70de5e4c75ac2b2aa47d62af13fed8c4d7c6e3f938e2d932
Oct 30 21:52:04.903762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.903651    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm" event={"ID":"28fb0f05-db53-49e3-8d70-9aa2df134292","Type":"ContainerStarted","Data":"1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685"}
Oct 30 21:52:04.903762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.903724    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm" event={"ID":"28fb0f05-db53-49e3-8d70-9aa2df134292","Type":"ContainerStarted","Data":"dd6b15e147e1605a70de5e4c75ac2b2aa47d62af13fed8c4d7c6e3f938e2d932"}
Oct 30 21:52:04.903762 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.903751    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:04.906663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.906056    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r" event={"ID":"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda","Type":"ContainerStarted","Data":"b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756"}
Oct 30 21:52:04.906663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.906120    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:04.906663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.906143    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r" event={"ID":"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda","Type":"ContainerStarted","Data":"d2740c6f27649f70839d249d85ac1b2ec335b03ec7568733aa219235af247fae"}
Oct 30 21:52:04.949673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.949603    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm" podStartSLOduration=1.949581694 podStartE2EDuration="1.949581694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:04.945354152 +0000 UTC m=+3107.719754092" watchObservedRunningTime="2026-10-30 21:52:04.949581694 +0000 UTC m=+3107.723981634"
Oct 30 21:52:04.987316 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:04.987230    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r" podStartSLOduration=1.9872103399999999 podStartE2EDuration="1.98721034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:04.984366935 +0000 UTC m=+3107.758766875" watchObservedRunningTime="2026-10-30 21:52:04.98721034 +0000 UTC m=+3107.761610282"
Oct 30 21:52:05.908661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:05.908613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:07.359359 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:07.359328    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:09.402121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:09.402087    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:10.746702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:10.746669    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:10.877536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:10.877502    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:12.364857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:12.364806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:14.423387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:14.423351    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:15.893046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:15.893005    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:15.915698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:15.915669    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:52:15.915901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:15.915822    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:52:17.332541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:17.332508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:18.825142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:18.825109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:20.298285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:20.298254    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:20.656674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:20.656635    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:21.222855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:21.199888    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:21.375221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:21.375189    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:22.027675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:22.027635    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:23.835981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:23.835947    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:25.446349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:25.446308    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:25.604052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:25.603993    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:26.077166 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:26.077120    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:26.213525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:26.213483    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:26.333700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:26.333613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:26.656626 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:52:26.656534    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:52:27.120710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:27.120667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:27.270428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:27.270298    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:27.517856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:27.517704    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:29.425997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:29.425950    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:30.993860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:30.993798    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:32.477213 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:32.477173    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:34.229152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:34.229115    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:35.785807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:35.785753    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:37.848084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:37.848046    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:39.451450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:39.451416    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:41.033015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:41.032977    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:42.530867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:42.530795    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:44.321488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:44.321433    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:45.774381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:45.774334    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:45.911893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:45.911286    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:47.387018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:47.386985    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:48.449641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:48.449601    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b_a198c7a6-5911-4d40-aa9d-4877f4d67ad1/payload-processing/0.log"
Oct 30 21:52:48.631013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:48.630980    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 21:52:48.827938 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:48.827910    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 21:52:49.017271 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:49.017245    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg_a4111e34-86a0-4dd6-a4f7-f602e71d41eb/payload-processing/0.log"
Oct 30 21:52:49.208085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:49.208005    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg_a4111e34-86a0-4dd6-a4f7-f602e71d41eb/payload-processing/0.log"
Oct 30 21:52:49.324079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:49.324042    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:50.905032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:50.904971    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.234769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.234733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.322425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.322391    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"]
Oct 30 21:52:52.322643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.322623    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6" podUID="c6161306-e9d9-4740-9538-74358591c4af" containerName="payload-processing" containerID="cri-o://5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f" gracePeriod=30
Oct 30 21:52:52.342090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.342058    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"]
Oct 30 21:52:52.342309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.342284    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2" podUID="45822463-ab68-460b-9e1e-de6937b9835e" containerName="payload-pre-processing" containerID="cri-o://a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db" gracePeriod=30
Oct 30 21:52:52.421656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.421618    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.542668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.542482    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.668381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.668202    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.845923 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.845892    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:52:52.850030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.850007    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:52:52.932424 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.932372    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.969091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969054    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w\" (UniqueName: \"kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w\") pod \"c6161306-e9d9-4740-9538-74358591c4af\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") "
Oct 30 21:52:52.969320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969123    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume\") pod \"c6161306-e9d9-4740-9538-74358591c4af\" (UID: \"c6161306-e9d9-4740-9538-74358591c4af\") "
Oct 30 21:52:52.969320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969192    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume\") pod \"45822463-ab68-460b-9e1e-de6937b9835e\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") "
Oct 30 21:52:52.969508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969343    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc\" (UniqueName: \"kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc\") pod \"45822463-ab68-460b-9e1e-de6937b9835e\" (UID: \"45822463-ab68-460b-9e1e-de6937b9835e\") "
Oct 30 21:52:52.969747 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969718    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume" pod "45822463-ab68-460b-9e1e-de6937b9835e" (UID: "45822463-ab68-460b-9e1e-de6937b9835e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:52.969990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969964    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume" pod "c6161306-e9d9-4740-9538-74358591c4af" (UID: "c6161306-e9d9-4740-9538-74358591c4af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:52.969990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.969986    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45822463-ab68-460b-9e1e-de6937b9835e-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:52:52.971786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.971757    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc" pod "45822463-ab68-460b-9e1e-de6937b9835e" (UID: "45822463-ab68-460b-9e1e-de6937b9835e"). InnerVolumeSpecName "kube-api-access-22kcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:52.972156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:52.972121    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w" pod "c6161306-e9d9-4740-9538-74358591c4af" (UID: "c6161306-e9d9-4740-9538-74358591c4af"). InnerVolumeSpecName "kube-api-access-64w9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:53.062109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.062067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:53.071653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.071623    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22kcc\" (UniqueName: \"kubernetes.io/projected/45822463-ab68-460b-9e1e-de6937b9835e-kube-api-access-22kcc\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:52:53.071653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.071653    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64w9w\" (UniqueName: \"kubernetes.io/projected/c6161306-e9d9-4740-9538-74358591c4af-kube-api-access-64w9w\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:52:53.071853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.071663    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6161306-e9d9-4740-9538-74358591c4af-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:52:53.186081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.185973    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:53.295871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.295796    4549 generic.go:356] "Generic (PLEG): container finished" podID="45822463-ab68-460b-9e1e-de6937b9835e" containerID="a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db" exitCode=0
Oct 30 21:52:53.296386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.295883    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"
Oct 30 21:52:53.296386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.295886    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2" event={"ID":"45822463-ab68-460b-9e1e-de6937b9835e","Type":"ContainerDied","Data":"a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db"}
Oct 30 21:52:53.296386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.295955    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2" event={"ID":"45822463-ab68-460b-9e1e-de6937b9835e","Type":"ContainerDied","Data":"dff737301eb85595845e531f5b0045a650d4c88d37526dd554d75343d931403f"}
Oct 30 21:52:53.296386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.295985    4549 scope.go:122] "RemoveContainer" containerID="a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db"
Oct 30 21:52:53.303718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.303678    4549 generic.go:356] "Generic (PLEG): container finished" podID="c6161306-e9d9-4740-9538-74358591c4af" containerID="5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f" exitCode=0
Oct 30 21:52:53.303911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.303807    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6" event={"ID":"c6161306-e9d9-4740-9538-74358591c4af","Type":"ContainerDied","Data":"5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f"}
Oct 30 21:52:53.303911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.303866    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"
Oct 30 21:52:53.303911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.303882    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6" event={"ID":"c6161306-e9d9-4740-9538-74358591c4af","Type":"ContainerDied","Data":"fba4b4d54dc41742de60c3560dc825839d869bf37c355e88988ebe2fa8c8a1c1"}
Oct 30 21:52:53.313232 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.313121    4549 scope.go:122] "RemoveContainer" containerID="a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db"
Oct 30 21:52:53.313897 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:52:53.313824    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db\": container with ID starting with a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db not found: ID does not exist" containerID="a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db"
Oct 30 21:52:53.314033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.313912    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db"} err="failed to get container status \"a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db\": rpc error: code = NotFound desc = could not find container \"a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db\": container with ID starting with a0059fa46183bca92d63dbe0c01d3d02de7a7fb0997813d370510bb6788e26db not found: ID does not exist"
Oct 30 21:52:53.314033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.313937    4549 scope.go:122] "RemoveContainer" containerID="5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f"
Oct 30 21:52:53.327417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.327193    4549 scope.go:122] "RemoveContainer" containerID="5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f"
Oct 30 21:52:53.327607 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:52:53.327585    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f\": container with ID starting with 5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f not found: ID does not exist" containerID="5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f"
Oct 30 21:52:53.327671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.327617    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f"} err="failed to get container status \"5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f\": rpc error: code = NotFound desc = could not find container \"5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f\": container with ID starting with 5488ad7055265a67afa41e2b6b983dbf62bf0cb206c9d0f7cf241f826a079d5f not found: ID does not exist"
Oct 30 21:52:53.353671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.353639    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"]
Oct 30 21:52:53.358403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.358373    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2"]
Oct 30 21:52:53.408590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.408556    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"]
Oct 30 21:52:53.417482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.417450    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6"]
Oct 30 21:52:53.550622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:53.550526    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:54.793034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:54.792987    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:55.352524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:55.352483    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45822463-ab68-460b-9e1e-de6937b9835e" path="/var/lib/kubelet/pods/45822463-ab68-460b-9e1e-de6937b9835e/volumes"
Oct 30 21:52:55.353026 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:55.353010    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6161306-e9d9-4740-9538-74358591c4af" path="/var/lib/kubelet/pods/c6161306-e9d9-4740-9538-74358591c4af/volumes"
Oct 30 21:52:56.387764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:56.387726    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:56.575853 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:52:56.575802    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:52:58.315115 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:58.315083    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:59.878708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:52:59.878668    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:00.022178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:00.022129    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:00.174051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:00.173956    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"]
Oct 30 21:53:00.174640 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:00.174576    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" containerID="cri-o://b961d1a4b419b79d26a7ff166c047cf37aab55750ff0c31363e22af2438eac25" gracePeriod=30
Oct 30 21:53:00.295725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:00.295686    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:01.610212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:01.610174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:02.984104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:02.984068    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:04.572252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:04.572213    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:04.641804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:04.641764    4549 patch_prober.go:29] interesting pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:53:04.642029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:04.641859    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:53:05.442263 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.442232    4549 generic.go:356] "Generic (PLEG): container finished" podID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerID="b961d1a4b419b79d26a7ff166c047cf37aab55750ff0c31363e22af2438eac25" exitCode=0
Oct 30 21:53:05.442403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.442295    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" event={"ID":"71c031ff-19b0-42ba-9ba3-f9049f270dbb","Type":"ContainerDied","Data":"b961d1a4b419b79d26a7ff166c047cf37aab55750ff0c31363e22af2438eac25"}
Oct 30 21:53:05.442403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.442380    4549 scope.go:122] "RemoveContainer" containerID="c906afc1ca3ebdba8d04b12523881b6ce5c1b5cf73b21bbc15193179780c7178"
Oct 30 21:53:05.483764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.483740    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:53:05.671965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.671924    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.671965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.671968    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.671992    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672035    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672145    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672198    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672238    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672289    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672347    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672366    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672552    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:05.672570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672561    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data\") pod \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\" (UID: \"71c031ff-19b0-42ba-9ba3-f9049f270dbb\") "
Oct 30 21:53:05.673362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672651    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:05.673362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.672857    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:05.673486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.673397    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.673486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.673423    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.673486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.673438    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.673486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.673451    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.673486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.673463    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.674517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.674485    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:53:05.674712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.674689    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:05.674785 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.674709    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:05.675197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.675178    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw" pod "71c031ff-19b0-42ba-9ba3-f9049f270dbb" (UID: "71c031ff-19b0-42ba-9ba3-f9049f270dbb"). InnerVolumeSpecName "kube-api-access-x6zkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:05.774551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.774466    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6zkw\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-kube-api-access-x6zkw\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.774551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.774502    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.774551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.774512    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.774551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.774521    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c031ff-19b0-42ba-9ba3-f9049f270dbb-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:05.847783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.847738    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:05.988109 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:05.988075    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.216015    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"]
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217213    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c6161306-e9d9-4740-9538-74358591c4af" containerName="payload-processing"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217247    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6161306-e9d9-4740-9538-74358591c4af" containerName="payload-processing"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217283    4549 cpu_manager.go:414] "containerMap: removing container" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217293    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217325    4549 cpu_manager.go:414] "containerMap: removing container" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217332    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217394    4549 cpu_manager.go:414] "containerMap: removing container" podUID="45822463-ab68-460b-9e1e-de6937b9835e" containerName="payload-pre-processing"
Oct 30 21:53:06.218299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.217403    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45822463-ab68-460b-9e1e-de6937b9835e" containerName="payload-pre-processing"
Oct 30 21:53:06.230897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.222111    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6161306-e9d9-4740-9538-74358591c4af" containerName="payload-processing"
Oct 30 21:53:06.230897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.222151    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy"
Oct 30 21:53:06.230897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.222167    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" containerName="istio-proxy"
Oct 30 21:53:06.230897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.222177    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="45822463-ab68-460b-9e1e-de6937b9835e" containerName="payload-pre-processing"
Oct 30 21:53:06.230897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.228498    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.250926 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.235886    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"]
Oct 30 21:53:06.256688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.254300    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e136b8\""
Oct 30 21:53:06.256688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.255014    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e136b8-dockercfg-rl9vq\""
Oct 30 21:53:06.284845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.284786    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.285239 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.285209    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fllh\" (UniqueName: \"kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh\") pod \"payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.386690 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.386641    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.386922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.386871    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fllh\" (UniqueName: \"kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh\") pod \"payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.387622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.387605    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.394619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.394590    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"]
Oct 30 21:53:06.399186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.399166    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.415417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.415387    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fllh\" (UniqueName: \"kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh\") pod \"payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.423439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.423404    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"]
Oct 30 21:53:06.459383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.459354    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"
Oct 30 21:53:06.459668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.459442    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt" event={"ID":"71c031ff-19b0-42ba-9ba3-f9049f270dbb","Type":"ContainerDied","Data":"2ebec0ad910f0ef00b5db6e0ed8aca296a2052e7854cd9a37557ff5732c7f005"}
Oct 30 21:53:06.459810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.459709    4549 scope.go:122] "RemoveContainer" containerID="b961d1a4b419b79d26a7ff166c047cf37aab55750ff0c31363e22af2438eac25"
Oct 30 21:53:06.488018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.487981    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrwph\" (UniqueName: \"kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph\") pod \"payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.488201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.488181    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.543365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.543325    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"]
Oct 30 21:53:06.547694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.547657    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt"]
Oct 30 21:53:06.576133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.576095    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:06.589889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.589853    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.590040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.590026    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrwph\" (UniqueName: \"kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph\") pod \"payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.590723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.590694    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.612422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.612387    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrwph\" (UniqueName: \"kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph\") pod \"payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.714388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.714343    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:06.848349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.848304    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:06.906001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:06.898805    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"]
Oct 30 21:53:06.923110 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:53:06.923068    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b17d55d_9969_414c_80e8_56a95f97300c.slice/crio-035416cf9166466f87d99aabe7ecd79467b8e143f4ce122fd889daa760c5117c WatchSource:0}: Error finding container 035416cf9166466f87d99aabe7ecd79467b8e143f4ce122fd889daa760c5117c: Status 404 returned error can't find the container with id 035416cf9166466f87d99aabe7ecd79467b8e143f4ce122fd889daa760c5117c
Oct 30 21:53:07.025428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.025392    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"]
Oct 30 21:53:07.027125 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:53:07.027096    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98542825_adbc_4b11_85c2_9a589d543dfe.slice/crio-22a47939ebc3d0ae4e50b7257f84d5101d469b17891d2ae986216aa83c484399 WatchSource:0}: Error finding container 22a47939ebc3d0ae4e50b7257f84d5101d469b17891d2ae986216aa83c484399: Status 404 returned error can't find the container with id 22a47939ebc3d0ae4e50b7257f84d5101d469b17891d2ae986216aa83c484399
Oct 30 21:53:07.346932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.346886    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71c031ff-19b0-42ba-9ba3-f9049f270dbb" path="/var/lib/kubelet/pods/71c031ff-19b0-42ba-9ba3-f9049f270dbb/volumes"
Oct 30 21:53:07.468864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.468809    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4" event={"ID":"98542825-adbc-4b11-85c2-9a589d543dfe","Type":"ContainerStarted","Data":"3fc43a247cdd75b9c3fb7911375821452c23ab14420adf618c8b8cbd9db1469d"}
Oct 30 21:53:07.469072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.468892    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4" event={"ID":"98542825-adbc-4b11-85c2-9a589d543dfe","Type":"ContainerStarted","Data":"22a47939ebc3d0ae4e50b7257f84d5101d469b17891d2ae986216aa83c484399"}
Oct 30 21:53:07.469072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.468924    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:07.470680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.470651    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh" event={"ID":"2b17d55d-9969-414c-80e8-56a95f97300c","Type":"ContainerStarted","Data":"86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf"}
Oct 30 21:53:07.470865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.470708    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh" event={"ID":"2b17d55d-9969-414c-80e8-56a95f97300c","Type":"ContainerStarted","Data":"035416cf9166466f87d99aabe7ecd79467b8e143f4ce122fd889daa760c5117c"}
Oct 30 21:53:07.470865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.470738    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:07.515912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.515801    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4" podStartSLOduration=1.515781154 podStartE2EDuration="1.515781154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:07.513667975 +0000 UTC m=+3170.288067917" watchObservedRunningTime="2026-10-30 21:53:07.515781154 +0000 UTC m=+3170.290181094"
Oct 30 21:53:07.558441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:07.558380    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh" podStartSLOduration=1.558361222 podStartE2EDuration="1.558361222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:07.553640434 +0000 UTC m=+3170.328040386" watchObservedRunningTime="2026-10-30 21:53:07.558361222 +0000 UTC m=+3170.332761163"
Oct 30 21:53:08.749330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:08.749284    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:10.186997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:10.186965    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:10.311477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:10.311401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:10.437114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:10.437028    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:10.874974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:10.874921    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:11.801602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:11.801558    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:13.317929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:13.317894    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:15.237791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.237752    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:15.886361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.886321    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"]
Oct 30 21:53:15.892507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.892451    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.907360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.907319    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-f5bef0-openshift-default-dockercfg-rgcfl\""
Oct 30 21:53:15.923713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.923660    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"]
Oct 30 21:53:15.932856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.932800    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.933168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933137    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.933319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933217    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.933389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933361    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9m9s\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.933751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933723    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.933898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933764    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.933991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933924    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.934053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.933992    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:15.934142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:15.934115    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035033    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9m9s\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035111    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035131    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035161    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035265 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035220    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035284    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035339    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035392    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035484 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035451    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035715    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035746    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035794    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.035965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.035852    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.036133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.036113    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.037811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.037788    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.038027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.038011    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.055846 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.055798    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.055980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.055951    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9m9s\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s\") pod \"e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") " pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.186161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.186075    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:16.208162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.208122    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:16.528255 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:53:16.528224    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74aba30e_2e9d_4dc1_a75b_36aa0e70bc6e.slice/crio-3ba4ec398645fb14138777661172f4514de80118512657907b51bdd9f6f1c761 WatchSource:0}: Error finding container 3ba4ec398645fb14138777661172f4514de80118512657907b51bdd9f6f1c761: Status 404 returned error can't find the container with id 3ba4ec398645fb14138777661172f4514de80118512657907b51bdd9f6f1c761
Oct 30 21:53:16.529133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.529107    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"]
Oct 30 21:53:16.531636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.531595    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:16.531708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.531687    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:16.531755 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.531727    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:16.538889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.538859    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" event={"ID":"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e","Type":"ContainerStarted","Data":"3ba4ec398645fb14138777661172f4514de80118512657907b51bdd9f6f1c761"}
Oct 30 21:53:16.805236 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:16.805161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:17.547630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:17.547584    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" event={"ID":"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e","Type":"ContainerStarted","Data":"e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91"}
Oct 30 21:53:17.614856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:17.614769    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podStartSLOduration=2.6147553009999998 podStartE2EDuration="2.614755301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:17.601583651 +0000 UTC m=+3180.375983591" watchObservedRunningTime="2026-10-30 21:53:17.614755301 +0000 UTC m=+3180.389155242"
Oct 30 21:53:18.209152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:18.209108    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:18.210813 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:18.210780    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:53:18.210996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:18.210874    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused"
Oct 30 21:53:18.317622 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:18.317587    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:18.509917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:18.509050    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:53:18.526859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:18.523376    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:53:19.208893 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:19.208816    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:53:19.209432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:19.208907    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused"
Oct 30 21:53:19.884807 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:19.884762    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:20.224912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:20.212617    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:53:20.224912 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:20.212686    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused"
Oct 30 21:53:21.209046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:21.209003    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:53:21.209269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:21.209069    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused"
Oct 30 21:53:21.620995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:21.620904    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:22.208751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:22.208708    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused" start-of-body=
Oct 30 21:53:22.208998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:22.208788    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.218:15021/healthz/ready\": dial tcp 10.133.0.218:15021: connect: connection refused"
Oct 30 21:53:23.116659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:23.116626    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:23.484518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:23.484433    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:23.485197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:23.485167    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:23.485808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:23.485790    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:53:24.810949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:24.810912    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.434974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:26.434933    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.578036 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:53:26.577992    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:53:28.510869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:28.496360    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:30.027320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:30.027249    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:31.570849 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:31.570789    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.507053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:33.507013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.901254 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:33.901222    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:34.056186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:34.056154    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:34.193431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:34.193345    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:35.088706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:35.088666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:36.525511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:36.525476    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:38.138284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:38.138241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:39.643039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:39.642983    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:41.476683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:41.476625    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:42.931820 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:42.931784    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:44.494162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:44.494115    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:45.888694 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:45.888657    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:47.362044 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:47.362012    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:49.428391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:49.428352    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:51.030718 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:51.030680    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:52.638259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:52.638217    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:54.213681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:54.213641    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:55.600686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.600646    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:55.667544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.667505    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"]
Oct 30 21:53:55.667737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.667717    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs" podUID="a961251f-78ad-400f-8e1e-f57158b2dbfb" containerName="payload-processing" containerID="cri-o://e5ff07f43d824e964b08f938dc60774387304c4866a69e1cd649e1750ab765ee" gracePeriod=30
Oct 30 21:53:55.685302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.685266    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"]
Oct 30 21:53:55.685748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.685673    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2" podUID="ea1c02ef-de31-43d2-a814-b359ea86f26e" containerName="payload-pre-processing" containerID="cri-o://4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4" gracePeriod=30
Oct 30 21:53:55.778200 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.778168    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:55.926540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:55.926411    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:56.107195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.104094    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:53:56.151732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.151693    4549 generic.go:356] "Generic (PLEG): container finished" podID="a961251f-78ad-400f-8e1e-f57158b2dbfb" containerID="e5ff07f43d824e964b08f938dc60774387304c4866a69e1cd649e1750ab765ee" exitCode=0
Oct 30 21:53:56.151965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.151799    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs" event={"ID":"a961251f-78ad-400f-8e1e-f57158b2dbfb","Type":"ContainerDied","Data":"e5ff07f43d824e964b08f938dc60774387304c4866a69e1cd649e1750ab765ee"}
Oct 30 21:53:56.156723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.156690    4549 generic.go:356] "Generic (PLEG): container finished" podID="ea1c02ef-de31-43d2-a814-b359ea86f26e" containerID="4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4" exitCode=0
Oct 30 21:53:56.157025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.156986    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2" event={"ID":"ea1c02ef-de31-43d2-a814-b359ea86f26e","Type":"ContainerDied","Data":"4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4"}
Oct 30 21:53:56.157194 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.157178    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2" event={"ID":"ea1c02ef-de31-43d2-a814-b359ea86f26e","Type":"ContainerDied","Data":"5d83e11d23911a969710461b1a3e0dd95ea6fcfcdca0d2721a668554b984b88f"}
Oct 30 21:53:56.157331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.157319    4549 scope.go:122] "RemoveContainer" containerID="4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4"
Oct 30 21:53:56.157757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.157738    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"
Oct 30 21:53:56.161520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.161496    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:56.165251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.165230    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:53:56.180089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.180061    4549 scope.go:122] "RemoveContainer" containerID="4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4"
Oct 30 21:53:56.180493 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:53:56.180466    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4\": container with ID starting with 4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4 not found: ID does not exist" containerID="4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4"
Oct 30 21:53:56.180607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.180501    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4"} err="failed to get container status \"4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4\": rpc error: code = NotFound desc = could not find container \"4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4\": container with ID starting with 4f9d237ae0aac5623a493ae6ed0bbb4c20e7777bd51199af5740cdef22927ff4 not found: ID does not exist"
Oct 30 21:53:56.211749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.211718    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx\" (UniqueName: \"kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx\") pod \"ea1c02ef-de31-43d2-a814-b359ea86f26e\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") "
Oct 30 21:53:56.211998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.211937    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume\") pod \"ea1c02ef-de31-43d2-a814-b359ea86f26e\" (UID: \"ea1c02ef-de31-43d2-a814-b359ea86f26e\") "
Oct 30 21:53:56.212411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.212385    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume" pod "ea1c02ef-de31-43d2-a814-b359ea86f26e" (UID: "ea1c02ef-de31-43d2-a814-b359ea86f26e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:56.212545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.212528    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea1c02ef-de31-43d2-a814-b359ea86f26e-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:56.216139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.215075    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx" pod "ea1c02ef-de31-43d2-a814-b359ea86f26e" (UID: "ea1c02ef-de31-43d2-a814-b359ea86f26e"). InnerVolumeSpecName "kube-api-access-cwrzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:56.294280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.294241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:56.313183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.313152    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume\") pod \"a961251f-78ad-400f-8e1e-f57158b2dbfb\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") "
Oct 30 21:53:56.313370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.313210    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42\" (UniqueName: \"kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42\") pod \"a961251f-78ad-400f-8e1e-f57158b2dbfb\" (UID: \"a961251f-78ad-400f-8e1e-f57158b2dbfb\") "
Oct 30 21:53:56.313647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.313611    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume" pod "a961251f-78ad-400f-8e1e-f57158b2dbfb" (UID: "a961251f-78ad-400f-8e1e-f57158b2dbfb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:56.313757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.313620    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwrzx\" (UniqueName: \"kubernetes.io/projected/ea1c02ef-de31-43d2-a814-b359ea86f26e-kube-api-access-cwrzx\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:56.315717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.315695    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42" pod "a961251f-78ad-400f-8e1e-f57158b2dbfb" (UID: "a961251f-78ad-400f-8e1e-f57158b2dbfb"). InnerVolumeSpecName "kube-api-access-wpz42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:56.415686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.415551    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a961251f-78ad-400f-8e1e-f57158b2dbfb-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:56.415686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.415585    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpz42\" (UniqueName: \"kubernetes.io/projected/a961251f-78ad-400f-8e1e-f57158b2dbfb-kube-api-access-wpz42\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:53:56.416032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.415687    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:56.505797 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.505756    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"]
Oct 30 21:53:56.517343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.517312    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2"]
Oct 30 21:53:56.575370 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:53:56.575310    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:53:56.794183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:56.794147    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:57.165050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.165009    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs" event={"ID":"a961251f-78ad-400f-8e1e-f57158b2dbfb","Type":"ContainerDied","Data":"ab05467cbc3f22c42a0f40f6b2cb3a6066df2b1fb3059d3193db4a438d035c91"}
Oct 30 21:53:57.165272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.165100    4549 scope.go:122] "RemoveContainer" containerID="e5ff07f43d824e964b08f938dc60774387304c4866a69e1cd649e1750ab765ee"
Oct 30 21:53:57.165272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.165020    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"
Oct 30 21:53:57.218501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.218462    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"]
Oct 30 21:53:57.230465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.230427    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs"]
Oct 30 21:53:57.346567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.346536    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a961251f-78ad-400f-8e1e-f57158b2dbfb" path="/var/lib/kubelet/pods/a961251f-78ad-400f-8e1e-f57158b2dbfb/volumes"
Oct 30 21:53:57.347019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.347004    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea1c02ef-de31-43d2-a814-b359ea86f26e" path="/var/lib/kubelet/pods/ea1c02ef-de31-43d2-a814-b359ea86f26e/volumes"
Oct 30 21:53:57.564429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.564325    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:57.711037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.710420    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:57.840312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:57.840272    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:58.032913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:58.032876    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:59.581822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:53:59.581790    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:01.112661 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:01.112621    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.494708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.494672    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.568358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.568323    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"]
Oct 30 21:54:02.568576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.568554    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b" podUID="a198c7a6-5911-4d40-aa9d-4877f4d67ad1" containerName="payload-processing" containerID="cri-o://6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b" gracePeriod=30
Oct 30 21:54:02.582305 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.582262    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"]
Oct 30 21:54:02.584382 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.582621    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn" podUID="6d6292ef-6847-4ec0-9f25-52e6530dcdd5" containerName="payload-pre-processing" containerID="cri-o://b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5" gracePeriod=30
Oct 30 21:54:02.665752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.665697    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.795444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:02.795397    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:03.052623 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.052563    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:54:03.059346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.059317    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:54:03.090461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.090423    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl\" (UniqueName: \"kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl\") pod \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") "
Oct 30 21:54:03.090684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.090472    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume\") pod \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\" (UID: \"a198c7a6-5911-4d40-aa9d-4877f4d67ad1\") "
Oct 30 21:54:03.090684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.090537    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume\") pod \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") "
Oct 30 21:54:03.090684 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.090556    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp\" (UniqueName: \"kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp\") pod \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\" (UID: \"6d6292ef-6847-4ec0-9f25-52e6530dcdd5\") "
Oct 30 21:54:03.091472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.091439    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume" pod "a198c7a6-5911-4d40-aa9d-4877f4d67ad1" (UID: "a198c7a6-5911-4d40-aa9d-4877f4d67ad1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:03.092401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.092375    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume" pod "6d6292ef-6847-4ec0-9f25-52e6530dcdd5" (UID: "6d6292ef-6847-4ec0-9f25-52e6530dcdd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:03.094107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.094082    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl" pod "a198c7a6-5911-4d40-aa9d-4877f4d67ad1" (UID: "a198c7a6-5911-4d40-aa9d-4877f4d67ad1"). InnerVolumeSpecName "kube-api-access-jcqzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:03.095042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.095007    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp" pod "6d6292ef-6847-4ec0-9f25-52e6530dcdd5" (UID: "6d6292ef-6847-4ec0-9f25-52e6530dcdd5"). InnerVolumeSpecName "kube-api-access-9wflp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:03.158841 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.158780    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:03.192624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.192588    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcqzl\" (UniqueName: \"kubernetes.io/projected/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-kube-api-access-jcqzl\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:03.192624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.192621    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a198c7a6-5911-4d40-aa9d-4877f4d67ad1-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:03.192874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.192634    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:03.192874 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.192643    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wflp\" (UniqueName: \"kubernetes.io/projected/6d6292ef-6847-4ec0-9f25-52e6530dcdd5-kube-api-access-9wflp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:03.225418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.225379    4549 generic.go:356] "Generic (PLEG): container finished" podID="a198c7a6-5911-4d40-aa9d-4877f4d67ad1" containerID="6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b" exitCode=0
Oct 30 21:54:03.225619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.225440    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"
Oct 30 21:54:03.225619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.225470    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b" event={"ID":"a198c7a6-5911-4d40-aa9d-4877f4d67ad1","Type":"ContainerDied","Data":"6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b"}
Oct 30 21:54:03.225619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.225533    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b" event={"ID":"a198c7a6-5911-4d40-aa9d-4877f4d67ad1","Type":"ContainerDied","Data":"2245753fe639bdbc2389f6cb0d6fd73e3b8920d9d1ec35793282a239ff97ed37"}
Oct 30 21:54:03.225619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.225558    4549 scope.go:122] "RemoveContainer" containerID="6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b"
Oct 30 21:54:03.227192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.227162    4549 generic.go:356] "Generic (PLEG): container finished" podID="6d6292ef-6847-4ec0-9f25-52e6530dcdd5" containerID="b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5" exitCode=0
Oct 30 21:54:03.227312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.227300    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"
Oct 30 21:54:03.227360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.227322    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn" event={"ID":"6d6292ef-6847-4ec0-9f25-52e6530dcdd5","Type":"ContainerDied","Data":"b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5"}
Oct 30 21:54:03.227400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.227371    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn" event={"ID":"6d6292ef-6847-4ec0-9f25-52e6530dcdd5","Type":"ContainerDied","Data":"7963d83dbca5a407f41b40005429b94e2e6f14664137de1d526849a0813144b5"}
Oct 30 21:54:03.238408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.238383    4549 scope.go:122] "RemoveContainer" containerID="6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b"
Oct 30 21:54:03.238759 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:03.238738    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b\": container with ID starting with 6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b not found: ID does not exist" containerID="6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b"
Oct 30 21:54:03.238882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.238774    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b"} err="failed to get container status \"6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b\": rpc error: code = NotFound desc = could not find container \"6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b\": container with ID starting with 6db479a41fb5f5ab1084893e19be835cb02d4a3accac528559577838eaf8eb0b not found: ID does not exist"
Oct 30 21:54:03.238882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.238800    4549 scope.go:122] "RemoveContainer" containerID="b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5"
Oct 30 21:54:03.250031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.250008    4549 scope.go:122] "RemoveContainer" containerID="b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5"
Oct 30 21:54:03.250374 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:03.250355    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5\": container with ID starting with b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5 not found: ID does not exist" containerID="b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5"
Oct 30 21:54:03.250423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.250383    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5"} err="failed to get container status \"b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5\": rpc error: code = NotFound desc = could not find container \"b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5\": container with ID starting with b5d709ddfdd2be16e186e79501678fcf6303e359293a1bbeef7d4b5543b479a5 not found: ID does not exist"
Oct 30 21:54:03.282182 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.282102    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"]
Oct 30 21:54:03.287310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.287276    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b"]
Oct 30 21:54:03.357854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.355125    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:03.375275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.375234    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a198c7a6-5911-4d40-aa9d-4877f4d67ad1" path="/var/lib/kubelet/pods/a198c7a6-5911-4d40-aa9d-4877f4d67ad1/volumes"
Oct 30 21:54:03.376030 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.376000    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"]
Oct 30 21:54:03.376178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.376053    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn"]
Oct 30 21:54:03.720329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.720296    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:03.855409 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:03.855316    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:04.052118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:04.052080    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"]
Oct 30 21:54:04.052406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:04.052372    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy" containerID="cri-o://e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4" gracePeriod=30
Oct 30 21:54:04.113639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:04.113530    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:05.064241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:05.064202    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:05.345850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:05.345805    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d6292ef-6847-4ec0-9f25-52e6530dcdd5" path="/var/lib/kubelet/pods/6d6292ef-6847-4ec0-9f25-52e6530dcdd5/volumes"
Oct 30 21:54:06.626663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:06.626628    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:07.847222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:07.847179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:07.892611 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:07.892578    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"]
Oct 30 21:54:07.892909 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:07.892854    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2" podUID="6e762a22-f651-4111-b54a-40f89f8afdad" containerName="payload-processing" containerID="cri-o://a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa" gracePeriod=30
Oct 30 21:54:08.021320 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.021254    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.148224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.148130    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.197221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.197195    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:54:08.270965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.269657    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt\" (UniqueName: \"kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt\") pod \"6e762a22-f651-4111-b54a-40f89f8afdad\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") "
Oct 30 21:54:08.270965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.269786    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume\") pod \"6e762a22-f651-4111-b54a-40f89f8afdad\" (UID: \"6e762a22-f651-4111-b54a-40f89f8afdad\") "
Oct 30 21:54:08.270965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.270864    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume" pod "6e762a22-f651-4111-b54a-40f89f8afdad" (UID: "6e762a22-f651-4111-b54a-40f89f8afdad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:08.273814 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.273778    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt" pod "6e762a22-f651-4111-b54a-40f89f8afdad" (UID: "6e762a22-f651-4111-b54a-40f89f8afdad"). InnerVolumeSpecName "kube-api-access-pbrmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:08.375237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.371566    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.418093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.412574    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e762a22-f651-4111-b54a-40f89f8afdad-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:08.418093 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.412611    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbrmt\" (UniqueName: \"kubernetes.io/projected/6e762a22-f651-4111-b54a-40f89f8afdad-kube-api-access-pbrmt\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:08.449914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.448298    4549 generic.go:356] "Generic (PLEG): container finished" podID="6e762a22-f651-4111-b54a-40f89f8afdad" containerID="a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa" exitCode=0
Oct 30 21:54:08.449914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.448366    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2" event={"ID":"6e762a22-f651-4111-b54a-40f89f8afdad","Type":"ContainerDied","Data":"a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa"}
Oct 30 21:54:08.449914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.448420    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2" event={"ID":"6e762a22-f651-4111-b54a-40f89f8afdad","Type":"ContainerDied","Data":"70482c8206fee364f936b77c25fd613cb018ecb431503a37bdd5d5e46a72d0e3"}
Oct 30 21:54:08.449914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.448451    4549 scope.go:122] "RemoveContainer" containerID="a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa"
Oct 30 21:54:08.449914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.449504    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"
Oct 30 21:54:08.463940 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.463916    4549 scope.go:122] "RemoveContainer" containerID="a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa"
Oct 30 21:54:08.464871 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:08.464283    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa\": container with ID starting with a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa not found: ID does not exist" containerID="a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa"
Oct 30 21:54:08.464871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.464311    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa"} err="failed to get container status \"a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa\": rpc error: code = NotFound desc = could not find container \"a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa\": container with ID starting with a8597901bc24a488e4bb86aeb1318d4f007c7a1ddc57e0e9414a59895f5811aa not found: ID does not exist"
Oct 30 21:54:08.518739 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.518690    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"]
Oct 30 21:54:08.544981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.540912    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2"]
Oct 30 21:54:08.619310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.619278    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.800682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.800581    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.949474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:08.949436    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:09.091999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.091970    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:09.342168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.342139    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:54:09.347252 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.347218    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e762a22-f651-4111-b54a-40f89f8afdad" path="/var/lib/kubelet/pods/6e762a22-f651-4111-b54a-40f89f8afdad/volumes"
Oct 30 21:54:09.425315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425281    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.425315 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425321    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.425601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425395    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.425601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425415    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.425601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425469    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.425601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425568    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.426007 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.425982    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.426120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.426104    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.426199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.426142    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy\") pod \"0a182167-356a-4b2c-b874-eb0ea9057f39\" (UID: \"0a182167-356a-4b2c-b874-eb0ea9057f39\") "
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.426583    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.426773    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.426892    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.426966    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.427466    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.427488    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.427502    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a182167-356a-4b2c-b874-eb0ea9057f39-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.427901 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.427515    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.428711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.428581    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:09.428969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.428922    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:09.429389 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.429344    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "kube-api-access-d8cs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:09.431006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.430963    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:09.431562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.431533    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token" pod "0a182167-356a-4b2c-b874-eb0ea9057f39" (UID: "0a182167-356a-4b2c-b874-eb0ea9057f39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:09.465383 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.465348    4549 generic.go:356] "Generic (PLEG): container finished" podID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerID="e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4" exitCode=0
Oct 30 21:54:09.465591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.465408    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"
Oct 30 21:54:09.465591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.465422    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" event={"ID":"0a182167-356a-4b2c-b874-eb0ea9057f39","Type":"ContainerDied","Data":"e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4"}
Oct 30 21:54:09.465591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.465489    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29" event={"ID":"0a182167-356a-4b2c-b874-eb0ea9057f39","Type":"ContainerDied","Data":"a45deb11d5c816adeb58a5e2c03636b0853e7928a03d45f67107c61fff33020b"}
Oct 30 21:54:09.465591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.465522    4549 scope.go:122] "RemoveContainer" containerID="e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4"
Oct 30 21:54:09.477433 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.477413    4549 scope.go:122] "RemoveContainer" containerID="77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"
Oct 30 21:54:09.487999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.487979    4549 scope.go:122] "RemoveContainer" containerID="e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4"
Oct 30 21:54:09.488314 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:09.488291    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4\": container with ID starting with e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4 not found: ID does not exist" containerID="e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4"
Oct 30 21:54:09.488361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.488326    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4"} err="failed to get container status \"e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4\": rpc error: code = NotFound desc = could not find container \"e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4\": container with ID starting with e5ad07fe380b0c0ed2fa14bc553241b03dbb9fe8e7a967a2cd4f8e351cd91ee4 not found: ID does not exist"
Oct 30 21:54:09.488361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.488344    4549 scope.go:122] "RemoveContainer" containerID="77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"
Oct 30 21:54:09.488699 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:09.488681    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6\": container with ID starting with 77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6 not found: ID does not exist" containerID="77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"
Oct 30 21:54:09.488779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.488711    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6"} err="failed to get container status \"77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6\": rpc error: code = NotFound desc = could not find container \"77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6\": container with ID starting with 77814396c6d214a59bbc5c8137b2158b513683ba14fd6f4d8fd0001bf6984ce6 not found: ID does not exist"
Oct 30 21:54:09.520072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.520032    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"]
Oct 30 21:54:09.526312 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.526281    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29"]
Oct 30 21:54:09.529157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.529131    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.529279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.529162    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.529279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.529180    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.529279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.529190    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8cs5\" (UniqueName: \"kubernetes.io/projected/0a182167-356a-4b2c-b874-eb0ea9057f39-kube-api-access-d8cs5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:09.529279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:09.529198    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a182167-356a-4b2c-b874-eb0ea9057f39-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:10.270190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:10.270143    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:10.985090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:10.985059    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:11.346025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.345984    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" path="/var/lib/kubelet/pods/0a182167-356a-4b2c-b874-eb0ea9057f39/volumes"
Oct 30 21:54:11.402070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.402039    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:11.570968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.570934    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:11.767412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.761564    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"]
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775414    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ea1c02ef-de31-43d2-a814-b359ea86f26e" containerName="payload-pre-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775448    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea1c02ef-de31-43d2-a814-b359ea86f26e" containerName="payload-pre-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775491    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775500    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775524    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6e762a22-f651-4111-b54a-40f89f8afdad" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775532    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e762a22-f651-4111-b54a-40f89f8afdad" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775548    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6d6292ef-6847-4ec0-9f25-52e6530dcdd5" containerName="payload-pre-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775557    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d6292ef-6847-4ec0-9f25-52e6530dcdd5" containerName="payload-pre-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775576    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a961251f-78ad-400f-8e1e-f57158b2dbfb" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775583    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a961251f-78ad-400f-8e1e-f57158b2dbfb" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775601    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a198c7a6-5911-4d40-aa9d-4877f4d67ad1" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775608    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a198c7a6-5911-4d40-aa9d-4877f4d67ad1" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775636    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775644    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775843    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775864    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e762a22-f651-4111-b54a-40f89f8afdad" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775874    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d6292ef-6847-4ec0-9f25-52e6530dcdd5" containerName="payload-pre-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775886    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a961251f-78ad-400f-8e1e-f57158b2dbfb" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775899    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a198c7a6-5911-4d40-aa9d-4877f4d67ad1" containerName="payload-processing"
Oct 30 21:54:11.779897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.775913    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea1c02ef-de31-43d2-a814-b359ea86f26e" containerName="payload-pre-processing"
Oct 30 21:54:11.784521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.784168    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"]
Oct 30 21:54:11.784521 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.784297    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:11.815847 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.815794    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-be3ada-dockercfg-dr58v\""
Oct 30 21:54:11.816047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.815904    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-be3ada\""
Oct 30 21:54:11.860499 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.860382    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6wxn\" (UniqueName: \"kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn\") pod \"payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:11.860752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.860558    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:11.942913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.942874    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"]
Oct 30 21:54:11.943717 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.943694    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a182167-356a-4b2c-b874-eb0ea9057f39" containerName="istio-proxy"
Oct 30 21:54:11.947326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.947304    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:11.961367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.961340    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:11.961510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.961489    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6wxn\" (UniqueName: \"kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn\") pod \"payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:11.961996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.961975    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:11.967712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.967677    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"]
Oct 30 21:54:11.980703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:11.980673    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6wxn\" (UniqueName: \"kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn\") pod \"payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:12.062384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.062346    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmwc7\" (UniqueName: \"kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7\") pod \"payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.062564 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.062402    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.116777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.116679    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:12.163723 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.163687    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmwc7\" (UniqueName: \"kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7\") pod \"payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.163949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.163751    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.164465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.164443    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.186588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.186555    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmwc7\" (UniqueName: \"kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7\") pod \"payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.262143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.262101    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:12.421646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.421600    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"]
Oct 30 21:54:12.422900 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:54:12.422874    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb469b4b0_6207_42a6_b4e1_a89b457c975f.slice/crio-a9736133cac479b729e069f82dcd585886d1782bec94e78f634510e5cb36f030 WatchSource:0}: Error finding container a9736133cac479b729e069f82dcd585886d1782bec94e78f634510e5cb36f030: Status 404 returned error can't find the container with id a9736133cac479b729e069f82dcd585886d1782bec94e78f634510e5cb36f030
Oct 30 21:54:12.428400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.428367    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:12.506111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.506078    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" event={"ID":"b469b4b0-6207-42a6-b4e1-a89b457c975f","Type":"ContainerStarted","Data":"a9736133cac479b729e069f82dcd585886d1782bec94e78f634510e5cb36f030"}
Oct 30 21:54:12.569229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:12.569200    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"]
Oct 30 21:54:12.571104 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:54:12.571080    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70988b9_c257_47f2_98de_5eac5f217367.slice/crio-8dfd60a65c73812db7398922d514269006223f7d37969705b94ac7c807ea3764 WatchSource:0}: Error finding container 8dfd60a65c73812db7398922d514269006223f7d37969705b94ac7c807ea3764: Status 404 returned error can't find the container with id 8dfd60a65c73812db7398922d514269006223f7d37969705b94ac7c807ea3764
Oct 30 21:54:13.515274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.515234    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" event={"ID":"b469b4b0-6207-42a6-b4e1-a89b457c975f","Type":"ContainerStarted","Data":"5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51"}
Oct 30 21:54:13.515802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.515308    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:13.517439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.517401    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" event={"ID":"b70988b9-c257-47f2-98de-5eac5f217367","Type":"ContainerStarted","Data":"b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143"}
Oct 30 21:54:13.517546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.517454    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" event={"ID":"b70988b9-c257-47f2-98de-5eac5f217367","Type":"ContainerStarted","Data":"8dfd60a65c73812db7398922d514269006223f7d37969705b94ac7c807ea3764"}
Oct 30 21:54:13.517546 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.517483    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:13.557319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.557254    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" podStartSLOduration=2.557235665 podStartE2EDuration="2.557235665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:13.554809352 +0000 UTC m=+3236.329209292" watchObservedRunningTime="2026-10-30 21:54:13.557235665 +0000 UTC m=+3236.331635605"
Oct 30 21:54:13.598114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:13.598051    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" podStartSLOduration=2.598031059 podStartE2EDuration="2.598031059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:13.594609834 +0000 UTC m=+3236.369009773" watchObservedRunningTime="2026-10-30 21:54:13.598031059 +0000 UTC m=+3236.372431001"
Oct 30 21:54:14.317872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:14.317818    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:15.276581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:15.276543    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:15.477602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:15.477569    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:15.677504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:15.676218    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"]
Oct 30 21:54:15.677504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:15.676482    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy" containerID="cri-o://46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033" gracePeriod=30
Oct 30 21:54:15.741477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:15.741444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:15.960701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:15.960609    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:17.610196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:17.610161    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:19.109094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:19.109052    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:20.618052 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:20.617950    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:20.961726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:20.961698    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:54:21.105104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105072    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105195    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105258    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105317    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105465    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105505    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105570    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105428    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:21.105715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105508    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:21.105715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105544    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:21.105715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105627    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105652    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:21.105715 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105681    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token\") pod \"922ccaa2-5585-4c02-9305-8d2efe80d85f\" (UID: \"922ccaa2-5585-4c02-9305-8d2efe80d85f\") "
Oct 30 21:54:21.105984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.105714    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:21.106169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.106150    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.106242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.106176    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.106242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.106193    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.106242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.106206    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.106242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.106220    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922ccaa2-5585-4c02-9305-8d2efe80d85f-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.107879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.107852    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:21.107984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.107893    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:21.108264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.108245    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "kube-api-access-fsf4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:21.108325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.108244    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo" pod "922ccaa2-5585-4c02-9305-8d2efe80d85f" (UID: "922ccaa2-5585-4c02-9305-8d2efe80d85f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:21.207819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.207730    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.207819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.207763    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.207819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.207774    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.207819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.207783    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsf4d\" (UniqueName: \"kubernetes.io/projected/922ccaa2-5585-4c02-9305-8d2efe80d85f-kube-api-access-fsf4d\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:21.586659 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.586624    4549 generic.go:356] "Generic (PLEG): container finished" podID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerID="46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033" exitCode=0
Oct 30 21:54:21.586881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.586683    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"
Oct 30 21:54:21.586881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.586713    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" event={"ID":"922ccaa2-5585-4c02-9305-8d2efe80d85f","Type":"ContainerDied","Data":"46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033"}
Oct 30 21:54:21.586881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.586782    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5" event={"ID":"922ccaa2-5585-4c02-9305-8d2efe80d85f","Type":"ContainerDied","Data":"00e58af23f8f92ab969e1ae3942436e34a38a5f10b1e71296fc9588485bbafc5"}
Oct 30 21:54:21.586881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.586808    4549 scope.go:122] "RemoveContainer" containerID="46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033"
Oct 30 21:54:21.597469 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.597450    4549 scope.go:122] "RemoveContainer" containerID="4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"
Oct 30 21:54:21.607445 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.607422    4549 scope.go:122] "RemoveContainer" containerID="46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033"
Oct 30 21:54:21.607714 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:21.607695    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033\": container with ID starting with 46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033 not found: ID does not exist" containerID="46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033"
Oct 30 21:54:21.607753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.607723    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033"} err="failed to get container status \"46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033\": rpc error: code = NotFound desc = could not find container \"46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033\": container with ID starting with 46d13a9ae5791f95c71390ebfa2dbc579b8f2cb1f8da1caac41bbf8b766b0033 not found: ID does not exist"
Oct 30 21:54:21.607753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.607740    4549 scope.go:122] "RemoveContainer" containerID="4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"
Oct 30 21:54:21.608041 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:21.608021    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e\": container with ID starting with 4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e not found: ID does not exist" containerID="4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"
Oct 30 21:54:21.608084 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.608049    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e"} err="failed to get container status \"4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e\": rpc error: code = NotFound desc = could not find container \"4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e\": container with ID starting with 4c4e83c43494b02857dd9a5944dc71f37d56ace8f1bbe23c867302e9b991908e not found: ID does not exist"
Oct 30 21:54:21.638443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.638406    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"]
Oct 30 21:54:21.650420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:21.650394    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5"]
Oct 30 21:54:22.461229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:22.461195    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:23.345545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:23.345503    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" path="/var/lib/kubelet/pods/922ccaa2-5585-4c02-9305-8d2efe80d85f/volumes"
Oct 30 21:54:24.025404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:24.025363    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:24.526861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:24.526800    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:54:24.527291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:24.527260    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:54:25.235086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.235051    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"]
Oct 30 21:54:25.235298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.235252    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" podUID="c0b3d71c-3e38-46ed-9a74-71e70e7de322" containerName="maas-api" containerID="cri-o://5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406" gracePeriod=30
Oct 30 21:54:25.334518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.334488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:25.571992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.570056    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:54:25.628866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.618404    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca\") pod \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") "
Oct 30 21:54:25.628866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.618500    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\") pod \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") "
Oct 30 21:54:25.628866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.618685    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t\" (UniqueName: \"kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t\") pod \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") "
Oct 30 21:54:25.628866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.618739    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs\") pod \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\" (UID: \"c0b3d71c-3e38-46ed-9a74-71e70e7de322\") "
Oct 30 21:54:25.636862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.636412    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca" pod "c0b3d71c-3e38-46ed-9a74-71e70e7de322" (UID: "c0b3d71c-3e38-46ed-9a74-71e70e7de322"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:25.640002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.639962    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs" pod "c0b3d71c-3e38-46ed-9a74-71e70e7de322" (UID: "c0b3d71c-3e38-46ed-9a74-71e70e7de322"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:25.640341 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.640297    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls" pod "c0b3d71c-3e38-46ed-9a74-71e70e7de322" (UID: "c0b3d71c-3e38-46ed-9a74-71e70e7de322"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:25.642922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.642882    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t" pod "c0b3d71c-3e38-46ed-9a74-71e70e7de322" (UID: "c0b3d71c-3e38-46ed-9a74-71e70e7de322"). InnerVolumeSpecName "kube-api-access-dpt5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:25.654860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.653301    4549 generic.go:356] "Generic (PLEG): container finished" podID="c0b3d71c-3e38-46ed-9a74-71e70e7de322" containerID="5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406" exitCode=0
Oct 30 21:54:25.654860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.653510    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" event={"ID":"c0b3d71c-3e38-46ed-9a74-71e70e7de322","Type":"ContainerDied","Data":"5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406"}
Oct 30 21:54:25.654860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.653574    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh" event={"ID":"c0b3d71c-3e38-46ed-9a74-71e70e7de322","Type":"ContainerDied","Data":"12bdeebdc4dd10fd79b429336d556b85c18cde3a992f4eabd0ded57e7b99adb8"}
Oct 30 21:54:25.654860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.653607    4549 scope.go:122] "RemoveContainer" containerID="5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406"
Oct 30 21:54:25.654860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.653772    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"
Oct 30 21:54:25.691999 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.689440    4549 scope.go:122] "RemoveContainer" containerID="5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406"
Oct 30 21:54:25.692736 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:25.692692    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406\": container with ID starting with 5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406 not found: ID does not exist" containerID="5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406"
Oct 30 21:54:25.692986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.692941    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406"} err="failed to get container status \"5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406\": rpc error: code = NotFound desc = could not find container \"5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406\": container with ID starting with 5d056e49fa9bece1205c13da6e69f7b0cfc2745677b277cf2ab90fc226a74406 not found: ID does not exist"
Oct 30 21:54:25.749131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.749078    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"]
Oct 30 21:54:25.749347 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.749154    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ad73a4-df5c7888f-spsjh"]
Oct 30 21:54:25.751434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.751405    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0b3d71c-3e38-46ed-9a74-71e70e7de322-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:25.751434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.751437    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:25.751655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.751453    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpt5t\" (UniqueName: \"kubernetes.io/projected/c0b3d71c-3e38-46ed-9a74-71e70e7de322-kube-api-access-dpt5t\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:25.751655 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.751468    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0b3d71c-3e38-46ed-9a74-71e70e7de322-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:25.887612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:25.887570    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.046756 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:26.046694    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.326532 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:26.326491    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.501951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:26.501914    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.575753 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:26.575718    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:54:26.643627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:26.643599    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.023702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.023613    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"]
Oct 30 21:54:27.024928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024635    4549 cpu_manager.go:414] "containerMap: removing container" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy"
Oct 30 21:54:27.024928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024660    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy"
Oct 30 21:54:27.024928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024684    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c0b3d71c-3e38-46ed-9a74-71e70e7de322" containerName="maas-api"
Oct 30 21:54:27.024928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024691    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0b3d71c-3e38-46ed-9a74-71e70e7de322" containerName="maas-api"
Oct 30 21:54:27.024928 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024911    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy"
Oct 30 21:54:27.025191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024936    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy"
Oct 30 21:54:27.025191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.024947    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0b3d71c-3e38-46ed-9a74-71e70e7de322" containerName="maas-api"
Oct 30 21:54:27.030708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.030678    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.057176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.057147    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"]
Oct 30 21:54:27.062709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.062674    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-480d41-openshift-default-dockercfg-frhkk\""
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.209974    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48p52\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.210284    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.210458    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.210586    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.210615    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.210864    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.211018    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.211064    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.211899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.211223    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.217859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.212614    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.312764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.312667    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48p52\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.312764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.312748    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.313105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.312816    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.313105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.312862    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.313313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.313288    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.313757 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.313729    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.313974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.313821    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.314158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.314134    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.314237 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.314140    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.314641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.314617    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.316430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.316410    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.316623 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.316605    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.317667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.317646    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.318721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.316122    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.321866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.321820    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.337868 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.335920    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.358722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.352583    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.360823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.360763    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48p52\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52\") pod \"e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") " pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.366551 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.366519    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.369942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.369915    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0b3d71c-3e38-46ed-9a74-71e70e7de322" path="/var/lib/kubelet/pods/c0b3d71c-3e38-46ed-9a74-71e70e7de322/volumes"
Oct 30 21:54:27.526991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.526954    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.534703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.534664    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"]
Oct 30 21:54:27.534959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.534928    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4" podUID="98542825-adbc-4b11-85c2-9a589d543dfe" containerName="payload-processing" containerID="cri-o://3fc43a247cdd75b9c3fb7911375821452c23ab14420adf618c8b8cbd9db1469d" gracePeriod=30
Oct 30 21:54:27.555467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.555430    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"]
Oct 30 21:54:27.555733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.555699    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh" podUID="2b17d55d-9969-414c-80e8-56a95f97300c" containerName="payload-pre-processing" containerID="cri-o://86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf" gracePeriod=30
Oct 30 21:54:27.654039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.653999    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.654804 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.654785    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:27.672621 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.672591    4549 generic.go:356] "Generic (PLEG): container finished" podID="98542825-adbc-4b11-85c2-9a589d543dfe" containerID="3fc43a247cdd75b9c3fb7911375821452c23ab14420adf618c8b8cbd9db1469d" exitCode=0
Oct 30 21:54:27.672765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.672651    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4" event={"ID":"98542825-adbc-4b11-85c2-9a589d543dfe","Type":"ContainerDied","Data":"3fc43a247cdd75b9c3fb7911375821452c23ab14420adf618c8b8cbd9db1469d"}
Oct 30 21:54:27.786455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.786318    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.964435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:27.964229    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.006452 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.005941    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"]
Oct 30 21:54:28.009712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.009666    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:28.009853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.009768    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:28.009853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.009803    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:28.032691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.032650    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:54:28.036325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.036250    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:54:28.103990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.103952    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.212917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.212874    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh\" (UniqueName: \"kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh\") pod \"2b17d55d-9969-414c-80e8-56a95f97300c\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") "
Oct 30 21:54:28.213148 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.213021    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume\") pod \"98542825-adbc-4b11-85c2-9a589d543dfe\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") "
Oct 30 21:54:28.213223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.213173    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume\") pod \"2b17d55d-9969-414c-80e8-56a95f97300c\" (UID: \"2b17d55d-9969-414c-80e8-56a95f97300c\") "
Oct 30 21:54:28.213282 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.213262    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph\" (UniqueName: \"kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph\") pod \"98542825-adbc-4b11-85c2-9a589d543dfe\" (UID: \"98542825-adbc-4b11-85c2-9a589d543dfe\") "
Oct 30 21:54:28.214415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.214348    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume" pod "2b17d55d-9969-414c-80e8-56a95f97300c" (UID: "2b17d55d-9969-414c-80e8-56a95f97300c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:28.214793 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.214738    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume" pod "98542825-adbc-4b11-85c2-9a589d543dfe" (UID: "98542825-adbc-4b11-85c2-9a589d543dfe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:28.217380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.217347    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph" pod "98542825-adbc-4b11-85c2-9a589d543dfe" (UID: "98542825-adbc-4b11-85c2-9a589d543dfe"). InnerVolumeSpecName "kube-api-access-rrwph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:28.218815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.218778    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh" pod "2b17d55d-9969-414c-80e8-56a95f97300c" (UID: "2b17d55d-9969-414c-80e8-56a95f97300c"). InnerVolumeSpecName "kube-api-access-7fllh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:28.294122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.294083    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.315542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.315501    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98542825-adbc-4b11-85c2-9a589d543dfe-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:28.315542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.315535    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b17d55d-9969-414c-80e8-56a95f97300c-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:28.315542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.315545    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrwph\" (UniqueName: \"kubernetes.io/projected/98542825-adbc-4b11-85c2-9a589d543dfe-kube-api-access-rrwph\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:28.315542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.315554    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fllh\" (UniqueName: \"kubernetes.io/projected/2b17d55d-9969-414c-80e8-56a95f97300c-kube-api-access-7fllh\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:28.441724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.441668    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.601416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.601380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.680168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.680136    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"
Oct 30 21:54:28.680168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.680149    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4" event={"ID":"98542825-adbc-4b11-85c2-9a589d543dfe","Type":"ContainerDied","Data":"22a47939ebc3d0ae4e50b7257f84d5101d469b17891d2ae986216aa83c484399"}
Oct 30 21:54:28.680727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.680206    4549 scope.go:122] "RemoveContainer" containerID="3fc43a247cdd75b9c3fb7911375821452c23ab14420adf618c8b8cbd9db1469d"
Oct 30 21:54:28.682035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.681995    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" event={"ID":"e8adf648-d12f-4e04-95be-2b76723a4b97","Type":"ContainerStarted","Data":"6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"}
Oct 30 21:54:28.682158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.682061    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" event={"ID":"e8adf648-d12f-4e04-95be-2b76723a4b97","Type":"ContainerStarted","Data":"2f15c222b35bc3a480cfbcfedc12d0b0290030c46df2f95210b9df783b9222fa"}
Oct 30 21:54:28.683584 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.683560    4549 generic.go:356] "Generic (PLEG): container finished" podID="2b17d55d-9969-414c-80e8-56a95f97300c" containerID="86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf" exitCode=0
Oct 30 21:54:28.683701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.683607    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"
Oct 30 21:54:28.683701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.683635    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh" event={"ID":"2b17d55d-9969-414c-80e8-56a95f97300c","Type":"ContainerDied","Data":"86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf"}
Oct 30 21:54:28.683701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.683681    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh" event={"ID":"2b17d55d-9969-414c-80e8-56a95f97300c","Type":"ContainerDied","Data":"035416cf9166466f87d99aabe7ecd79467b8e143f4ce122fd889daa760c5117c"}
Oct 30 21:54:28.691482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.691460    4549 scope.go:122] "RemoveContainer" containerID="86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf"
Oct 30 21:54:28.702099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.702076    4549 scope.go:122] "RemoveContainer" containerID="86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf"
Oct 30 21:54:28.702419 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:28.702395    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf\": container with ID starting with 86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf not found: ID does not exist" containerID="86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf"
Oct 30 21:54:28.702472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.702431    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf"} err="failed to get container status \"86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf\": rpc error: code = NotFound desc = could not find container \"86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf\": container with ID starting with 86aeeb600c0ac0ebc910793f8637d146f63b97280422a62ffcf1cf199a25a5cf not found: ID does not exist"
Oct 30 21:54:28.752264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.752205    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podStartSLOduration=2.752188805 podStartE2EDuration="2.752188805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:28.74700522 +0000 UTC m=+3251.521405174" watchObservedRunningTime="2026-10-30 21:54:28.752188805 +0000 UTC m=+3251.526588743"
Oct 30 21:54:28.791122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.791086    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"]
Oct 30 21:54:28.794518 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.794485    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4"]
Oct 30 21:54:28.829143 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.829111    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"]
Oct 30 21:54:28.834931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:28.834902    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh"]
Oct 30 21:54:29.345349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.345316    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b17d55d-9969-414c-80e8-56a95f97300c" path="/var/lib/kubelet/pods/2b17d55d-9969-414c-80e8-56a95f97300c/volumes"
Oct 30 21:54:29.345873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.345817    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98542825-adbc-4b11-85c2-9a589d543dfe" path="/var/lib/kubelet/pods/98542825-adbc-4b11-85c2-9a589d543dfe/volumes"
Oct 30 21:54:29.655411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.655302    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:29.667259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.667214    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:29.693888 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.693810    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:29.695068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.695040    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:54:29.883990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:29.883959    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:31.302311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:31.302277    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:31.564910 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:31.563063    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:32.313682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:32.313640    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:32.980049 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:32.980006    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.066385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.066341    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"]
Oct 30 21:54:33.067060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067042    4549 cpu_manager.go:414] "containerMap: removing container" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy"
Oct 30 21:54:33.067119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067062    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="922ccaa2-5585-4c02-9305-8d2efe80d85f" containerName="istio-proxy"
Oct 30 21:54:33.067119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067074    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2b17d55d-9969-414c-80e8-56a95f97300c" containerName="payload-pre-processing"
Oct 30 21:54:33.067119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067080    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b17d55d-9969-414c-80e8-56a95f97300c" containerName="payload-pre-processing"
Oct 30 21:54:33.067119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067100    4549 cpu_manager.go:414] "containerMap: removing container" podUID="98542825-adbc-4b11-85c2-9a589d543dfe" containerName="payload-processing"
Oct 30 21:54:33.067119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067105    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98542825-adbc-4b11-85c2-9a589d543dfe" containerName="payload-processing"
Oct 30 21:54:33.067290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067216    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b17d55d-9969-414c-80e8-56a95f97300c" containerName="payload-pre-processing"
Oct 30 21:54:33.067290 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.067227    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="98542825-adbc-4b11-85c2-9a589d543dfe" containerName="payload-processing"
Oct 30 21:54:33.072298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.072276    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.094803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.092451    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"]
Oct 30 21:54:33.103015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.102984    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-480d41\""
Oct 30 21:54:33.103400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.103379    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-480d41-dockercfg-dqnsq\""
Oct 30 21:54:33.103677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.103644    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"]
Oct 30 21:54:33.108955 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.108911    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.115927 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.115806    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.129060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.128088    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"]
Oct 30 21:54:33.217075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.217017    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.217509 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.217483    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8l9t\" (UniqueName: \"kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t\") pod \"payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.217824 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.217806    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.218142 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.218085    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b\") pod \"payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.320351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.320308    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b\") pod \"payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.320768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.320423    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.320768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.320508    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8l9t\" (UniqueName: \"kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t\") pod \"payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.320768 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.320621    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.321667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.321644    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.322775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.322750    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.345119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.345080    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b\") pod \"payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.367106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.365289    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8l9t\" (UniqueName: \"kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t\") pod \"payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.403119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.403085    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:33.437096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.437056    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.443159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.443084    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:33.639877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.639118    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.730571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.730537    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"]
Oct 30 21:54:33.736010 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:54:33.733398    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode354e8ef_2a80_468c_874f_255626055aa1.slice/crio-c7473de77c0a30ffd2b4529ef2b139ae7696e75cbe87f7718b7b6c3a2711501f WatchSource:0}: Error finding container c7473de77c0a30ffd2b4529ef2b139ae7696e75cbe87f7718b7b6c3a2711501f: Status 404 returned error can't find the container with id c7473de77c0a30ffd2b4529ef2b139ae7696e75cbe87f7718b7b6c3a2711501f
Oct 30 21:54:33.860174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.856931    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"]
Oct 30 21:54:33.864790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.864741    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:33.896188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.895021    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f8137c\""
Oct 30 21:54:33.897583 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.897502    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"]
Oct 30 21:54:33.903736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.903704    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"]
Oct 30 21:54:33.907317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.906444    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f8137c-dockercfg-dtz2v\""
Oct 30 21:54:33.955585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.955543    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:33.955726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.955590    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24j6\" (UniqueName: \"kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6\") pod \"payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:33.992535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.992506    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"]
Oct 30 21:54:33.996752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:33.996726    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.016328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.016294    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"]
Oct 30 21:54:34.057257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.057203    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf2zv\" (UniqueName: \"kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv\") pod \"payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.057512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.057354    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:34.057512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.057416    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p24j6\" (UniqueName: \"kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6\") pod \"payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:34.057512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.057495    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.058345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.058313    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:34.078950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.078917    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24j6\" (UniqueName: \"kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6\") pod \"payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:34.159378 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.159274    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf2zv\" (UniqueName: \"kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv\") pod \"payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.159590 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.159424    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.160215 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.160189    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.179884 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.179846    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf2zv\" (UniqueName: \"kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv\") pod \"payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.210675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.210642    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:34.313151 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.313107    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.519700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.519668    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"]
Oct 30 21:54:34.520904 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:54:34.520869    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5898524_e326_4e39_bdce_3aaff9b2b5b9.slice/crio-55bb30b94c09438b515046782796e02083c8eb1f66f623df42a66c3c5a1c088e WatchSource:0}: Error finding container 55bb30b94c09438b515046782796e02083c8eb1f66f623df42a66c3c5a1c088e: Status 404 returned error can't find the container with id 55bb30b94c09438b515046782796e02083c8eb1f66f623df42a66c3c5a1c088e
Oct 30 21:54:34.525892 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.525864    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:34.640402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.640374    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"]
Oct 30 21:54:34.641806 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:54:34.641777    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67acd37b_ca60_4237_bfd6_bb38757d71bc.slice/crio-4f5202f20d82343a207c37b1164cea76f1ff48410c3e05c552f726f57a27f229 WatchSource:0}: Error finding container 4f5202f20d82343a207c37b1164cea76f1ff48410c3e05c552f726f57a27f229: Status 404 returned error can't find the container with id 4f5202f20d82343a207c37b1164cea76f1ff48410c3e05c552f726f57a27f229
Oct 30 21:54:34.739446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.739412    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq" event={"ID":"2cc788fd-f20b-45b4-aac3-6c50a7f3f367","Type":"ContainerStarted","Data":"51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53"}
Oct 30 21:54:34.739628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.739476    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq" event={"ID":"2cc788fd-f20b-45b4-aac3-6c50a7f3f367","Type":"ContainerStarted","Data":"c7e7d4aba09588322cb47db01e9e74b422f0cc7e2096fbdc5ffaedc3b0e3cd40"}
Oct 30 21:54:34.739699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.739632    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:34.741226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.741193    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn" event={"ID":"b5898524-e326-4e39-bdce-3aaff9b2b5b9","Type":"ContainerStarted","Data":"e06d672d79ba06ad29f0cf7556aaae09c7f6002f4e79acdd7202636092476f07"}
Oct 30 21:54:34.741405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.741251    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn" event={"ID":"b5898524-e326-4e39-bdce-3aaff9b2b5b9","Type":"ContainerStarted","Data":"55bb30b94c09438b515046782796e02083c8eb1f66f623df42a66c3c5a1c088e"}
Oct 30 21:54:34.741405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.741285    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:34.742922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.742891    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54" event={"ID":"67acd37b-ca60-4237-bfd6-bb38757d71bc","Type":"ContainerStarted","Data":"131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef"}
Oct 30 21:54:34.743069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.742947    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54" event={"ID":"67acd37b-ca60-4237-bfd6-bb38757d71bc","Type":"ContainerStarted","Data":"4f5202f20d82343a207c37b1164cea76f1ff48410c3e05c552f726f57a27f229"}
Oct 30 21:54:34.743069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.742970    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:34.744726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.744703    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq" event={"ID":"e354e8ef-2a80-468c-874f-255626055aa1","Type":"ContainerStarted","Data":"04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a"}
Oct 30 21:54:34.744823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.744749    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq" event={"ID":"e354e8ef-2a80-468c-874f-255626055aa1","Type":"ContainerStarted","Data":"c7473de77c0a30ffd2b4529ef2b139ae7696e75cbe87f7718b7b6c3a2711501f"}
Oct 30 21:54:34.744898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.744816    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:34.781458 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.781332    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq" podStartSLOduration=1.7813124660000001 podStartE2EDuration="1.781312466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:34.780028528 +0000 UTC m=+3257.554428481" watchObservedRunningTime="2026-10-30 21:54:34.781312466 +0000 UTC m=+3257.555712405"
Oct 30 21:54:34.826635 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.826581    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq" podStartSLOduration=1.826566463 podStartE2EDuration="1.826566463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:34.822549094 +0000 UTC m=+3257.596949033" watchObservedRunningTime="2026-10-30 21:54:34.826566463 +0000 UTC m=+3257.600966434"
Oct 30 21:54:34.866175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.866110    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn" podStartSLOduration=1.866091612 podStartE2EDuration="1.866091612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:34.864750581 +0000 UTC m=+3257.639150522" watchObservedRunningTime="2026-10-30 21:54:34.866091612 +0000 UTC m=+3257.640491567"
Oct 30 21:54:34.903297 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:34.903232    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54" podStartSLOduration=1.903214722 podStartE2EDuration="1.903214722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:34.902639239 +0000 UTC m=+3257.677039179" watchObservedRunningTime="2026-10-30 21:54:34.903214722 +0000 UTC m=+3257.677614663"
Oct 30 21:54:35.281531 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:35.281492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:35.528783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:35.528706    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:35.609327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:35.609291    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"]
Oct 30 21:54:35.609595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:35.609531    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy" containerID="cri-o://8478a026758460e6f9440444ef0dc34486838a5941f62ad7a7b03a9d153ad7ae" gracePeriod=30
Oct 30 21:54:35.786895 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:35.783223    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:36.204035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:36.203987    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"]
Oct 30 21:54:36.250703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:36.250653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:36.430103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:36.430067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:36.952795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:36.952764    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:37.107134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:37.107091    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:37.403374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:37.403325    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:38.135293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:38.135241    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:39.829107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:39.829075    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:40.823231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:40.823191    4549 generic.go:356] "Generic (PLEG): container finished" podID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerID="8478a026758460e6f9440444ef0dc34486838a5941f62ad7a7b03a9d153ad7ae" exitCode=0
Oct 30 21:54:40.823231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:40.823218    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" event={"ID":"ad3671de-b297-412e-a72b-3d54d7553e3d","Type":"ContainerDied","Data":"8478a026758460e6f9440444ef0dc34486838a5941f62ad7a7b03a9d153ad7ae"}
Oct 30 21:54:40.823492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:40.823313    4549 scope.go:122] "RemoveContainer" containerID="a82af0e983e1ea6bb8646f1aedf8e6efae2a0ccb0e0326e6821581fee2fbc0ee"
Oct 30 21:54:40.893512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:40.893448    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:54:41.007343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007310    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007371    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007391    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007454    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007563    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007638    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.007736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007688    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007736 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007724    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007714    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.007937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007766    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.007937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007774    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.007937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007790    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.007937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.007872    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp\") pod \"ad3671de-b297-412e-a72b-3d54d7553e3d\" (UID: \"ad3671de-b297-412e-a72b-3d54d7553e3d\") "
Oct 30 21:54:41.008543 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.008491    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:41.008976 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.008946    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3671de-b297-412e-a72b-3d54d7553e3d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.009112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.008982    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.009112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.008998    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.009112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.009013    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.009112 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.009058    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.010286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.010265    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.010449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.010429    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "kube-api-access-mhlrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:41.010616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.010595    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:41.011087 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.011068    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token" pod "ad3671de-b297-412e-a72b-3d54d7553e3d" (UID: "ad3671de-b297-412e-a72b-3d54d7553e3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:41.110092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.110054    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.110092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.110089    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.110325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.110102    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3671de-b297-412e-a72b-3d54d7553e3d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.110325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.110118    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhlrp\" (UniqueName: \"kubernetes.io/projected/ad3671de-b297-412e-a72b-3d54d7553e3d-kube-api-access-mhlrp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:41.330181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.330143    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.831451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.831416    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv" event={"ID":"ad3671de-b297-412e-a72b-3d54d7553e3d","Type":"ContainerDied","Data":"a6b8a6422c135b8991dad5bf1e7c6f1583b92669d272035dec7168711f591aaf"}
Oct 30 21:54:41.831638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.831423    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"
Oct 30 21:54:41.831638 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.831484    4549 scope.go:122] "RemoveContainer" containerID="8478a026758460e6f9440444ef0dc34486838a5941f62ad7a7b03a9d153ad7ae"
Oct 30 21:54:41.906511 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.906474    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"]
Oct 30 21:54:41.918455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:41.918418    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv"]
Oct 30 21:54:42.941015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:42.940978    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:43.345343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:43.345303    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" path="/var/lib/kubelet/pods/ad3671de-b297-412e-a72b-3d54d7553e3d/volumes"
Oct 30 21:54:44.419700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:44.419663    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:45.657497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:45.657462    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:45.779697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:45.779664    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:54:45.780732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:45.780705    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:54:45.781613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:45.781593    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:54:45.782154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:45.782136    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:54:46.107627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:46.107594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.402028 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:46.401939    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.716601 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:46.716510    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.839667 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:46.839635    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.462555 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:47.462511    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.635486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:47.635446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.964506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:47.957060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:48.158921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:48.158047    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:48.301435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:48.301346    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:48.582467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:48.582430    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"]
Oct 30 21:54:48.620519 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:48.620477    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:48.890257 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:48.890170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:49.111921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:49.111881    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:50.501374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:50.501343    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:51.230273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.230235    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g" podUID="61b35dec-44ce-4347-aab1-2d19496c87de" containerName="main" containerID="cri-o://69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c" gracePeriod=45
Oct 30 21:54:51.517905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.517874    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:54:51.615742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.615709    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir\") pod \"61b35dec-44ce-4347-aab1-2d19496c87de\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") "
Oct 30 21:54:51.615952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.615780    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache\") pod \"61b35dec-44ce-4347-aab1-2d19496c87de\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") "
Oct 30 21:54:51.615952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.615926    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm\") pod \"61b35dec-44ce-4347-aab1-2d19496c87de\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") "
Oct 30 21:54:51.616111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.615959    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6\" (UniqueName: \"kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6\") pod \"61b35dec-44ce-4347-aab1-2d19496c87de\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") "
Oct 30 21:54:51.616111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616003    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs\" (UniqueName: \"kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs\") pod \"61b35dec-44ce-4347-aab1-2d19496c87de\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") "
Oct 30 21:54:51.616111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616083    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache" pod "61b35dec-44ce-4347-aab1-2d19496c87de" (UID: "61b35dec-44ce-4347-aab1-2d19496c87de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:51.616353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616090    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir" pod "61b35dec-44ce-4347-aab1-2d19496c87de" (UID: "61b35dec-44ce-4347-aab1-2d19496c87de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:51.616353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616130    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home\") pod \"61b35dec-44ce-4347-aab1-2d19496c87de\" (UID: \"61b35dec-44ce-4347-aab1-2d19496c87de\") "
Oct 30 21:54:51.616472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616358    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home" pod "61b35dec-44ce-4347-aab1-2d19496c87de" (UID: "61b35dec-44ce-4347-aab1-2d19496c87de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:51.616665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616633    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:51.616665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616657    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:51.616822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.616670    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:51.618617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.618594    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs" pod "61b35dec-44ce-4347-aab1-2d19496c87de" (UID: "61b35dec-44ce-4347-aab1-2d19496c87de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:51.618765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.618750    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm" pod "61b35dec-44ce-4347-aab1-2d19496c87de" (UID: "61b35dec-44ce-4347-aab1-2d19496c87de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:51.618811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.618791    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6" pod "61b35dec-44ce-4347-aab1-2d19496c87de" (UID: "61b35dec-44ce-4347-aab1-2d19496c87de"). InnerVolumeSpecName "kube-api-access-4cpm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:51.718023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.717981    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b35dec-44ce-4347-aab1-2d19496c87de-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:51.718023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.718018    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cpm6\" (UniqueName: \"kubernetes.io/projected/61b35dec-44ce-4347-aab1-2d19496c87de-kube-api-access-4cpm6\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:51.718023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.718031    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b35dec-44ce-4347-aab1-2d19496c87de-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:54:51.952345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.952305    4549 generic.go:356] "Generic (PLEG): container finished" podID="61b35dec-44ce-4347-aab1-2d19496c87de" containerID="69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c" exitCode=0
Oct 30 21:54:51.952563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.952374    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"
Oct 30 21:54:51.952563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.952382    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g" event={"ID":"61b35dec-44ce-4347-aab1-2d19496c87de","Type":"ContainerDied","Data":"69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c"}
Oct 30 21:54:51.952563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.952430    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g" event={"ID":"61b35dec-44ce-4347-aab1-2d19496c87de","Type":"ContainerDied","Data":"550d2c0fa4779e8e1d570d7eca2d4e0d03aaf55c00c11789516424606ade9c32"}
Oct 30 21:54:51.952563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.952449    4549 scope.go:122] "RemoveContainer" containerID="69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c"
Oct 30 21:54:51.964385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.964362    4549 scope.go:122] "RemoveContainer" containerID="69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c"
Oct 30 21:54:51.964699 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:51.964679    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c\": container with ID starting with 69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c not found: ID does not exist" containerID="69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c"
Oct 30 21:54:51.964798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:51.964708    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c"} err="failed to get container status \"69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c\": rpc error: code = NotFound desc = could not find container \"69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c\": container with ID starting with 69c04940b1c3bda3833eb6f4d32738176d29c53252e93ace2f577d91dea0561c not found: ID does not exist"
Oct 30 21:54:52.003865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:52.003813    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"]
Oct 30 21:54:52.015134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:52.015100    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w1-889195/sub-test-model-889195-kserve-cb5b88f45-6qx9g"]
Oct 30 21:54:52.827364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:52.827317    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:52.972751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:52.972708    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:53.350719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:53.350668    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61b35dec-44ce-4347-aab1-2d19496c87de" path="/var/lib/kubelet/pods/61b35dec-44ce-4347-aab1-2d19496c87de/volumes"
Oct 30 21:54:53.583679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:53.583632    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerName="payload-pre-processing" containerID="cri-o://05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c" gracePeriod=25
Oct 30 21:54:54.781552 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:54.781410    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:55.868145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:55.868113    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:56.057898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:56.057815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:56.294918 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:56.287869    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:56.555902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:56.555766    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:56.581076 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:54:56.581041    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:54:56.707240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:56.707204    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:58.398057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:54:58.398025    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:00.059503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:00.059466    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:01.575504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:01.575467    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"]
Oct 30 21:55:01.749812 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:01.749770    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:02.054370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:02.054273    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:03.266937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:03.266893    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:03.461943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:03.461912    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:04.703956 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:04.703918    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"]
Oct 30 21:55:04.704653 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:04.704160    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw" podUID="92b1f981-df5b-4f82-8330-be68205dd0b4" containerName="maas-api" containerID="cri-o://d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6" gracePeriod=30
Oct 30 21:55:04.808228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:04.808181    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.075612 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.075549    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:55:05.165097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.165049    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs\") pod \"92b1f981-df5b-4f82-8330-be68205dd0b4\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") "
Oct 30 21:55:05.165334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.165214    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf\" (UniqueName: \"kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf\") pod \"92b1f981-df5b-4f82-8330-be68205dd0b4\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") "
Oct 30 21:55:05.165419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.165397    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca\") pod \"92b1f981-df5b-4f82-8330-be68205dd0b4\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") "
Oct 30 21:55:05.165514 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.165498    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls\") pod \"92b1f981-df5b-4f82-8330-be68205dd0b4\" (UID: \"92b1f981-df5b-4f82-8330-be68205dd0b4\") "
Oct 30 21:55:05.168190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.168147    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca" pod "92b1f981-df5b-4f82-8330-be68205dd0b4" (UID: "92b1f981-df5b-4f82-8330-be68205dd0b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:05.176508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.176368    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls" pod "92b1f981-df5b-4f82-8330-be68205dd0b4" (UID: "92b1f981-df5b-4f82-8330-be68205dd0b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:05.179234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.179166    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf" pod "92b1f981-df5b-4f82-8330-be68205dd0b4" (UID: "92b1f981-df5b-4f82-8330-be68205dd0b4"). InnerVolumeSpecName "kube-api-access-4plgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:05.181754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.180979    4549 generic.go:356] "Generic (PLEG): container finished" podID="92b1f981-df5b-4f82-8330-be68205dd0b4" containerID="d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6" exitCode=0
Oct 30 21:55:05.181754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.181025    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw" event={"ID":"92b1f981-df5b-4f82-8330-be68205dd0b4","Type":"ContainerDied","Data":"d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6"}
Oct 30 21:55:05.181754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.181075    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw" event={"ID":"92b1f981-df5b-4f82-8330-be68205dd0b4","Type":"ContainerDied","Data":"11fd09b4658bcb23175ed0ec7a7a10ca071e9d770c9e19916c2aca49f902acb4"}
Oct 30 21:55:05.181754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.181111    4549 scope.go:122] "RemoveContainer" containerID="d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6"
Oct 30 21:55:05.181754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.181289    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"
Oct 30 21:55:05.183196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.183125    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs" pod "92b1f981-df5b-4f82-8330-be68205dd0b4" (UID: "92b1f981-df5b-4f82-8330-be68205dd0b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:05.215971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.215941    4549 scope.go:122] "RemoveContainer" containerID="d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6"
Oct 30 21:55:05.216627 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:05.216556    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6\": container with ID starting with d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6 not found: ID does not exist" containerID="d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6"
Oct 30 21:55:05.216627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.216593    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6"} err="failed to get container status \"d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6\": rpc error: code = NotFound desc = could not find container \"d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6\": container with ID starting with d719ba41fc1b519fe4b6edef32db5f6f889804fe407dcde43ffca05cf820a4a6 not found: ID does not exist"
Oct 30 21:55:05.267363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.267310    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b1f981-df5b-4f82-8330-be68205dd0b4-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:05.267363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.267358    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:05.267363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.267375    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b1f981-df5b-4f82-8330-be68205dd0b4-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:05.267771 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.267388    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4plgf\" (UniqueName: \"kubernetes.io/projected/92b1f981-df5b-4f82-8330-be68205dd0b4-kube-api-access-4plgf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:05.315628 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.315577    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.450224 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.450185    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.535403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.535363    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"]
Oct 30 21:55:05.539031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.539000    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-66cc54-84488746c5-dcrmw"]
Oct 30 21:55:05.609091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.609003    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.944147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:05.944059    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:07.320149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:07.320118    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"]
Oct 30 21:55:07.345247 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:07.345209    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92b1f981-df5b-4f82-8330-be68205dd0b4" path="/var/lib/kubelet/pods/92b1f981-df5b-4f82-8330-be68205dd0b4/volumes"
Oct 30 21:55:07.357149 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:07.357111    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"]
Oct 30 21:55:07.527720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:07.527681    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:07.682711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:07.682598    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:08.994498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.994463    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"]
Oct 30 21:55:08.995160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995146    4549 cpu_manager.go:414] "containerMap: removing container" podUID="61b35dec-44ce-4347-aab1-2d19496c87de" containerName="main"
Oct 30 21:55:08.995218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995161    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b35dec-44ce-4347-aab1-2d19496c87de" containerName="main"
Oct 30 21:55:08.995218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995177    4549 cpu_manager.go:414] "containerMap: removing container" podUID="92b1f981-df5b-4f82-8330-be68205dd0b4" containerName="maas-api"
Oct 30 21:55:08.995218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995182    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92b1f981-df5b-4f82-8330-be68205dd0b4" containerName="maas-api"
Oct 30 21:55:08.995218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995198    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy"
Oct 30 21:55:08.995218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995203    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy"
Oct 30 21:55:08.995218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995219    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy"
Oct 30 21:55:08.995412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995224    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy"
Oct 30 21:55:08.995412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995321    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b35dec-44ce-4347-aab1-2d19496c87de" containerName="main"
Oct 30 21:55:08.995412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995331    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="92b1f981-df5b-4f82-8330-be68205dd0b4" containerName="maas-api"
Oct 30 21:55:08.995412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995338    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy"
Oct 30 21:55:08.995412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:08.995345    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad3671de-b297-412e-a72b-3d54d7553e3d" containerName="istio-proxy"
Oct 30 21:55:09.000219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.000200    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.019929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.019896    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-480d41-serving-cert\""
Oct 30 21:55:09.040099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.040059    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"]
Oct 30 21:55:09.115085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.115044    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:09.118473 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.118446    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.118586 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.118521    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.118644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.118608    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctqvb\" (UniqueName: \"kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.118809 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.118790    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.220208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.220165    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.221275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.221250    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctqvb\" (UniqueName: \"kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.221975 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.221950    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.222998 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.222978    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.223645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.222865    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.231465 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.231434    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.245591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.245488    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.274766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.274720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:09.283131 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.283056    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctqvb\" (UniqueName: \"kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb\") pod \"maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.321248 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.321210    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:09.709860 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.709797    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"]
Oct 30 21:55:09.710806 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:55:09.710781    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08d2214_1986_49ed_8b92_687522c66890.slice/crio-088848be363c8355a79ce2ac41cdde0c7276c61e3254f782b282f9c6edbad1cf WatchSource:0}: Error finding container 088848be363c8355a79ce2ac41cdde0c7276c61e3254f782b282f9c6edbad1cf: Status 404 returned error can't find the container with id 088848be363c8355a79ce2ac41cdde0c7276c61e3254f782b282f9c6edbad1cf
Oct 30 21:55:09.712262 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.712245    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:55:09.838554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:09.838454    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:10.012496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:10.012416    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:10.264950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:10.251328    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:10.273981 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:10.273872    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7" event={"ID":"e08d2214-1986-49ed-8b92-687522c66890","Type":"ContainerStarted","Data":"6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57"}
Oct 30 21:55:10.274493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:10.274375    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7" event={"ID":"e08d2214-1986-49ed-8b92-687522c66890","Type":"ContainerStarted","Data":"088848be363c8355a79ce2ac41cdde0c7276c61e3254f782b282f9c6edbad1cf"}
Oct 30 21:55:10.276095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:10.276043    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:10.346434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:10.343698    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7" podStartSLOduration=2.006931664 podStartE2EDuration="2.343676424s" totalImagesPullingTime="336.74476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:10.338310127 +0000 UTC m=+3293.112710068" watchObservedRunningTime="2026-10-30 21:55:10.343676424 +0000 UTC m=+3293.118076364"
Oct 30 21:55:11.218033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:11.217996    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:11.395606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:11.395572    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:12.320752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:12.320708    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerName="payload-processing" containerID="cri-o://67e77be3824fe68c6a7210e6fa02a9c180ec62c3ad9b906e41e1180d724f7f2a" gracePeriod=25
Oct 30 21:55:12.358032 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:12.357998    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" containerName="payload-pre-processing" containerID="cri-o://9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58" gracePeriod=25
Oct 30 21:55:13.268272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:13.268230    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:13.346072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:13.346035    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:13.826352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:13.826315    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:13.938133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:13.938087    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:55:14.034777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.034740    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.044186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.044150    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume\") pod \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") "
Oct 30 21:55:14.044381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.044318    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg\" (UniqueName: \"kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg\") pod \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\" (UID: \"3c405a84-a806-4b83-a9ed-6b2eafd74fb8\") "
Oct 30 21:55:14.044545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.044521    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume" pod "3c405a84-a806-4b83-a9ed-6b2eafd74fb8" (UID: "3c405a84-a806-4b83-a9ed-6b2eafd74fb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:14.045055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.045034    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:14.046654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.046631    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg" pod "3c405a84-a806-4b83-a9ed-6b2eafd74fb8" (UID: "3c405a84-a806-4b83-a9ed-6b2eafd74fb8"). InnerVolumeSpecName "kube-api-access-ljlcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:14.146365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.146280    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljlcg\" (UniqueName: \"kubernetes.io/projected/3c405a84-a806-4b83-a9ed-6b2eafd74fb8-kube-api-access-ljlcg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:14.226764 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.226379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.341921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.339942    4549 generic.go:356] "Generic (PLEG): container finished" podID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerID="05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c" exitCode=0
Oct 30 21:55:14.341921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.340040    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" event={"ID":"3c405a84-a806-4b83-a9ed-6b2eafd74fb8","Type":"ContainerDied","Data":"05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c"}
Oct 30 21:55:14.341921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.340090    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv" event={"ID":"3c405a84-a806-4b83-a9ed-6b2eafd74fb8","Type":"ContainerDied","Data":"468fc0cff7471d04c10cf5ceb3ddfca2b1b21a223e0e88097033924a488a5a8f"}
Oct 30 21:55:14.341921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.340121    4549 scope.go:122] "RemoveContainer" containerID="05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c"
Oct 30 21:55:14.341921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.340335    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"
Oct 30 21:55:14.366046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.366018    4549 scope.go:122] "RemoveContainer" containerID="05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c"
Oct 30 21:55:14.366451 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:14.366375    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c\": container with ID starting with 05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c not found: ID does not exist" containerID="05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c"
Oct 30 21:55:14.366451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.366402    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c"} err="failed to get container status \"05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c\": rpc error: code = NotFound desc = could not find container \"05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c\": container with ID starting with 05c71b8df19e4fc7a7f0cfa868212fa44dce131d5e8f00559e2ee4b7f4171e9c not found: ID does not exist"
Oct 30 21:55:14.393432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.393392    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.406172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.406091    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"]
Oct 30 21:55:14.417420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.417386    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv"]
Oct 30 21:55:14.586303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.586271    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.735278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.735182    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:14.735278 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:14.735231    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:15.344883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:15.344847    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" path="/var/lib/kubelet/pods/3c405a84-a806-4b83-a9ed-6b2eafd74fb8/volumes"
Oct 30 21:55:16.197167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.196953    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:16.598894 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.598847    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main" containerID="cri-o://d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36" gracePeriod=45
Oct 30 21:55:16.874699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.874665    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:55:16.991013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.990982    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb\" (UniqueName: \"kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb\") pod \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") "
Oct 30 21:55:16.991193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991036    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm\") pod \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") "
Oct 30 21:55:16.991193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991150    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache\") pod \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") "
Oct 30 21:55:16.991291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991218    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir\") pod \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") "
Oct 30 21:55:16.991291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991268    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home\") pod \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") "
Oct 30 21:55:16.991403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991298    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs\") pod \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\" (UID: \"ff1362cb-2c95-45a4-8a59-6c7a2d335192\") "
Oct 30 21:55:16.991525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991444    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache" pod "ff1362cb-2c95-45a4-8a59-6c7a2d335192" (UID: "ff1362cb-2c95-45a4-8a59-6c7a2d335192"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:16.991646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991526    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir" pod "ff1362cb-2c95-45a4-8a59-6c7a2d335192" (UID: "ff1362cb-2c95-45a4-8a59-6c7a2d335192"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:16.991646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991589    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home" pod "ff1362cb-2c95-45a4-8a59-6c7a2d335192" (UID: "ff1362cb-2c95-45a4-8a59-6c7a2d335192"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:16.991915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991892    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:16.991985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991920    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:16.991985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.991936    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:16.993474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.993452    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb" pod "ff1362cb-2c95-45a4-8a59-6c7a2d335192" (UID: "ff1362cb-2c95-45a4-8a59-6c7a2d335192"). InnerVolumeSpecName "kube-api-access-frmsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:16.993627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.993613    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs" pod "ff1362cb-2c95-45a4-8a59-6c7a2d335192" (UID: "ff1362cb-2c95-45a4-8a59-6c7a2d335192"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:16.993724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:16.993711    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm" pod "ff1362cb-2c95-45a4-8a59-6c7a2d335192" (UID: "ff1362cb-2c95-45a4-8a59-6c7a2d335192"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:17.093369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.093331    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frmsb\" (UniqueName: \"kubernetes.io/projected/ff1362cb-2c95-45a4-8a59-6c7a2d335192-kube-api-access-frmsb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:17.093369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.093369    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff1362cb-2c95-45a4-8a59-6c7a2d335192-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:17.093646 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.093385    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff1362cb-2c95-45a4-8a59-6c7a2d335192-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:17.318367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.318342    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:55:17.361439 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.361407    4549 generic.go:356] "Generic (PLEG): container finished" podID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerID="d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36" exitCode=0
Oct 30 21:55:17.361557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.361443    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"
Oct 30 21:55:17.361557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.361481    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" event={"ID":"ff1362cb-2c95-45a4-8a59-6c7a2d335192","Type":"ContainerDied","Data":"d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36"}
Oct 30 21:55:17.361557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.361523    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks" event={"ID":"ff1362cb-2c95-45a4-8a59-6c7a2d335192","Type":"ContainerDied","Data":"0e3b0175ebbdf6a3915404758a5593ca53210a578ca69bac62e8f9a32ca8465e"}
Oct 30 21:55:17.361557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.361546    4549 scope.go:122] "RemoveContainer" containerID="d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36"
Oct 30 21:55:17.372986 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.372966    4549 scope.go:122] "RemoveContainer" containerID="d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36"
Oct 30 21:55:17.373314 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:17.373292    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36\": container with ID starting with d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36 not found: ID does not exist" containerID="d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36"
Oct 30 21:55:17.373399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.373325    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36"} err="failed to get container status \"d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36\": rpc error: code = NotFound desc = could not find container \"d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36\": container with ID starting with d300a54416cb5bce981a83aeaef2f406ae280c8ba59e6c2333c8516389c58b36 not found: ID does not exist"
Oct 30 21:55:17.417732 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.417697    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"]
Oct 30 21:55:17.427040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:17.427007    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-79477e/distinct-2-w3-79477e-kserve-59b74cb58d-wljks"]
Oct 30 21:55:18.205654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:18.205615    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.856873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:18.856822    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:55:18.868161 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:18.868139    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 21:55:19.344330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:19.344294    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" path="/var/lib/kubelet/pods/ff1362cb-2c95-45a4-8a59-6c7a2d335192/volumes"
Oct 30 21:55:19.890079 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:19.890038    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:20.369939 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:20.369902    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:21.486719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:21.486690    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:21.661455 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:21.661419    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:21.913317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:21.913278    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"]
Oct 30 21:55:21.913600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:21.913572    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" containerID="cri-o://e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91" gracePeriod=30
Oct 30 21:55:22.047748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:22.047710    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:23.488101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:23.488067    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:55:23.488525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:23.488132    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:55:23.624122 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:23.624067    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:24.736432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:24.736381    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:24.737227 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:24.737196    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:25.126536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:25.126503    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"]
Oct 30 21:55:25.181980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:25.181944    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:25.303054 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:25.303019    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:25.434513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:25.434424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:26.576246 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:26.576203    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:55:26.836012 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:26.835921    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:27.185598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.185563    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:55:27.231500 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231465    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231515    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231549    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231601    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231743    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231790    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231872    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.231924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231898    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.232171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231953    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:27.232171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.231962    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data\") pod \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\" (UID: \"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e\") "
Oct 30 21:55:27.232301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.232206    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:27.232364 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.232332    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:27.232591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.232443    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:27.232660 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.232620    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:27.233198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.233171    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.233328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.233203    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.233328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.233218    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.233328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.233235    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.233328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.233250    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.234102 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.234068    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:27.234205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.234162    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "kube-api-access-z9m9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:27.234205 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.234172    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:55:27.234339 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.234323    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token" pod "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" (UID: "74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:27.334313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.334269    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.334313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.334304    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9m9s\" (UniqueName: \"kubernetes.io/projected/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-kube-api-access-z9m9s\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.334313 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.334318    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.334576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.334329    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:27.428029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.427992    4549 generic.go:356] "Generic (PLEG): container finished" podID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerID="e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91" exitCode=0
Oct 30 21:55:27.428029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.428033    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"
Oct 30 21:55:27.428326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.428075    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" event={"ID":"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e","Type":"ContainerDied","Data":"e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91"}
Oct 30 21:55:27.428326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.428146    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl" event={"ID":"74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e","Type":"ContainerDied","Data":"3ba4ec398645fb14138777661172f4514de80118512657907b51bdd9f6f1c761"}
Oct 30 21:55:27.428326 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.428182    4549 scope.go:122] "RemoveContainer" containerID="e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91"
Oct 30 21:55:27.439430 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.439407    4549 scope.go:122] "RemoveContainer" containerID="e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91"
Oct 30 21:55:27.439795 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:27.439772    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91\": container with ID starting with e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91 not found: ID does not exist" containerID="e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91"
Oct 30 21:55:27.439879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.439807    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91"} err="failed to get container status \"e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91\": rpc error: code = NotFound desc = could not find container \"e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91\": container with ID starting with e68f6ef54f70906268c8c503ac0805de28f73cfee63e40507ac65085664c8e91 not found: ID does not exist"
Oct 30 21:55:27.483117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.483084    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"]
Oct 30 21:55:27.487266 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:27.487237    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl"]
Oct 30 21:55:28.512065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.512027    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:28.551674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.551638    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"]
Oct 30 21:55:28.551932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.551908    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm" podUID="28fb0f05-db53-49e3-8d70-9aa2df134292" containerName="payload-processing" containerID="cri-o://1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685" gracePeriod=30
Oct 30 21:55:28.582822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.582785    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"]
Oct 30 21:55:28.583153 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.583095    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r" podUID="bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" containerName="payload-pre-processing" containerID="cri-o://b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756" gracePeriod=30
Oct 30 21:55:28.665971 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.665931    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:28.891865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.879358    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:28.999096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:28.999059    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:55:29.029396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.029369    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:55:29.080615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.080521    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume\") pod \"28fb0f05-db53-49e3-8d70-9aa2df134292\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") "
Oct 30 21:55:29.080615 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.080587    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl\") pod \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") "
Oct 30 21:55:29.080924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.080702    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume\") pod \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\" (UID: \"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda\") "
Oct 30 21:55:29.080924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.080849    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk\" (UniqueName: \"kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk\") pod \"28fb0f05-db53-49e3-8d70-9aa2df134292\" (UID: \"28fb0f05-db53-49e3-8d70-9aa2df134292\") "
Oct 30 21:55:29.081060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.080991    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume" pod "28fb0f05-db53-49e3-8d70-9aa2df134292" (UID: "28fb0f05-db53-49e3-8d70-9aa2df134292"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:29.081375 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.081353    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28fb0f05-db53-49e3-8d70-9aa2df134292-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:29.081498 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.081408    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume" pod "bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" (UID: "bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:29.084520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.084494    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl" pod "bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" (UID: "bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda"). InnerVolumeSpecName "kube-api-access-sscwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:29.086006 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.085983    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk" pod "28fb0f05-db53-49e3-8d70-9aa2df134292" (UID: "28fb0f05-db53-49e3-8d70-9aa2df134292"). InnerVolumeSpecName "kube-api-access-qkzfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:29.172251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.172215    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:29.182641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.182613    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-kube-api-access-sscwl\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:29.182641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.182639    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:29.182880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.182651    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkzfk\" (UniqueName: \"kubernetes.io/projected/28fb0f05-db53-49e3-8d70-9aa2df134292-kube-api-access-qkzfk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:29.325733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.325700    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:29.357246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.357166    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" path="/var/lib/kubelet/pods/74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e/volumes"
Oct 30 21:55:29.449353 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.449312    4549 generic.go:356] "Generic (PLEG): container finished" podID="28fb0f05-db53-49e3-8d70-9aa2df134292" containerID="1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685" exitCode=0
Oct 30 21:55:29.449576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.449381    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"
Oct 30 21:55:29.449576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.449402    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm" event={"ID":"28fb0f05-db53-49e3-8d70-9aa2df134292","Type":"ContainerDied","Data":"1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685"}
Oct 30 21:55:29.449576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.449517    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm" event={"ID":"28fb0f05-db53-49e3-8d70-9aa2df134292","Type":"ContainerDied","Data":"dd6b15e147e1605a70de5e4c75ac2b2aa47d62af13fed8c4d7c6e3f938e2d932"}
Oct 30 21:55:29.449576 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.449551    4549 scope.go:122] "RemoveContainer" containerID="1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685"
Oct 30 21:55:29.452036 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.452005    4549 generic.go:356] "Generic (PLEG): container finished" podID="bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" containerID="b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756" exitCode=0
Oct 30 21:55:29.452390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.452168    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r" event={"ID":"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda","Type":"ContainerDied","Data":"b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756"}
Oct 30 21:55:29.452390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.452212    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r" event={"ID":"bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda","Type":"ContainerDied","Data":"d2740c6f27649f70839d249d85ac1b2ec335b03ec7568733aa219235af247fae"}
Oct 30 21:55:29.452390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.452300    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"
Oct 30 21:55:29.482882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.473180    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:29.488870 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.488848    4549 scope.go:122] "RemoveContainer" containerID="1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685"
Oct 30 21:55:29.489301 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:29.489268    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685\": container with ID starting with 1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685 not found: ID does not exist" containerID="1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685"
Oct 30 21:55:29.489423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.489311    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685"} err="failed to get container status \"1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685\": rpc error: code = NotFound desc = could not find container \"1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685\": container with ID starting with 1e96dccd3a2b7d7d9c08a272815f6470ef58bc201b4b2e9d7f6f7645f7139685 not found: ID does not exist"
Oct 30 21:55:29.489423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.489332    4549 scope.go:122] "RemoveContainer" containerID="b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756"
Oct 30 21:55:29.501191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.501172    4549 scope.go:122] "RemoveContainer" containerID="b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756"
Oct 30 21:55:29.501528 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:29.501506    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756\": container with ID starting with b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756 not found: ID does not exist" containerID="b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756"
Oct 30 21:55:29.501616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.501540    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756"} err="failed to get container status \"b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756\": rpc error: code = NotFound desc = could not find container \"b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756\": container with ID starting with b24adc46c3666f8192f30fafaabf15bf8d352b5bab6e8f4700315c8bd7145756 not found: ID does not exist"
Oct 30 21:55:29.525492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.525460    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"]
Oct 30 21:55:29.536372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.536343    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm"]
Oct 30 21:55:29.569201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.569162    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"]
Oct 30 21:55:29.575330 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.575304    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r"]
Oct 30 21:55:29.868945 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:29.868909    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:30.127218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:30.127107    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerName="payload-processing" containerID="cri-o://361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc" gracePeriod=25
Oct 30 21:55:31.097855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:31.097356    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:31.343654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:31.343623    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28fb0f05-db53-49e3-8d70-9aa2df134292" path="/var/lib/kubelet/pods/28fb0f05-db53-49e3-8d70-9aa2df134292/volumes"
Oct 30 21:55:31.344070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:31.344055    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" path="/var/lib/kubelet/pods/bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda/volumes"
Oct 30 21:55:32.502379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.502143    4549 generic.go:356] "Generic (PLEG): container finished" podID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerID="67e77be3824fe68c6a7210e6fa02a9c180ec62c3ad9b906e41e1180d724f7f2a" exitCode=0
Oct 30 21:55:32.502379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.502320    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" event={"ID":"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64","Type":"ContainerDied","Data":"67e77be3824fe68c6a7210e6fa02a9c180ec62c3ad9b906e41e1180d724f7f2a"}
Oct 30 21:55:32.706228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.706191    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:55:32.740634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.740604    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w\" (UniqueName: \"kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w\") pod \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") "
Oct 30 21:55:32.740822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.740690    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume\") pod \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\" (UID: \"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64\") "
Oct 30 21:55:32.741107 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.741078    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume" pod "5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" (UID: "5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:32.743059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.743034    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w" pod "5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" (UID: "5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64"). InnerVolumeSpecName "kube-api-access-ktl9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:32.754798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.754768    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:55:32.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.837383    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:32.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.847683    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume\") pod \"51eabe42-4d52-4337-ab89-418bc86f0849\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") "
Oct 30 21:55:32.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.848522    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h\" (UniqueName: \"kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h\") pod \"51eabe42-4d52-4337-ab89-418bc86f0849\" (UID: \"51eabe42-4d52-4337-ab89-418bc86f0849\") "
Oct 30 21:55:32.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.849350    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktl9w\" (UniqueName: \"kubernetes.io/projected/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-kube-api-access-ktl9w\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:32.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.849386    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:32.850854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.850137    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume" pod "51eabe42-4d52-4337-ab89-418bc86f0849" (UID: "51eabe42-4d52-4337-ab89-418bc86f0849"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:32.855867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.853590    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h" pod "51eabe42-4d52-4337-ab89-418bc86f0849" (UID: "51eabe42-4d52-4337-ab89-418bc86f0849"). InnerVolumeSpecName "kube-api-access-lzs2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:32.951235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.951155    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzs2h\" (UniqueName: \"kubernetes.io/projected/51eabe42-4d52-4337-ab89-418bc86f0849-kube-api-access-lzs2h\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:32.951235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:32.951183    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51eabe42-4d52-4337-ab89-418bc86f0849-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:33.146181 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.146150    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.306507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.306413    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.509549 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.509509    4549 generic.go:356] "Generic (PLEG): container finished" podID="51eabe42-4d52-4337-ab89-418bc86f0849" containerID="9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58" exitCode=0
Oct 30 21:55:33.510101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.509583    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"
Oct 30 21:55:33.510101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.509589    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" event={"ID":"51eabe42-4d52-4337-ab89-418bc86f0849","Type":"ContainerDied","Data":"9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58"}
Oct 30 21:55:33.510101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.509647    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z" event={"ID":"51eabe42-4d52-4337-ab89-418bc86f0849","Type":"ContainerDied","Data":"ccf95aa66790817fded629620b11f5badec63daf92a620fbdb387caabccaccf9"}
Oct 30 21:55:33.510101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.509669    4549 scope.go:122] "RemoveContainer" containerID="9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58"
Oct 30 21:55:33.511373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.511310    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"
Oct 30 21:55:33.511373 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.511326    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2" event={"ID":"5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64","Type":"ContainerDied","Data":"22db0dbd6b6daf65468497dcccec62b8ef8cc48ec87cae85eb386807d89f0ff3"}
Oct 30 21:55:33.519670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.519651    4549 scope.go:122] "RemoveContainer" containerID="9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58"
Oct 30 21:55:33.519990 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:33.519968    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58\": container with ID starting with 9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58 not found: ID does not exist" containerID="9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58"
Oct 30 21:55:33.520083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.520002    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58"} err="failed to get container status \"9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58\": rpc error: code = NotFound desc = could not find container \"9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58\": container with ID starting with 9fe7aa43c71590962fa6564544cfaf0fe601dba93c6c53952f21cf44f373de58 not found: ID does not exist"
Oct 30 21:55:33.520083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.520022    4549 scope.go:122] "RemoveContainer" containerID="67e77be3824fe68c6a7210e6fa02a9c180ec62c3ad9b906e41e1180d724f7f2a"
Oct 30 21:55:33.564256 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.563981    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"]
Oct 30 21:55:33.623682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.623519    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2"]
Oct 30 21:55:33.697857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.695761    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"]
Oct 30 21:55:33.711001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:33.710964    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z"]
Oct 30 21:55:34.256574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.256537    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.336357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.336324    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"]
Oct 30 21:55:34.336557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.336517    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg" podUID="a4111e34-86a0-4dd6-a4f7-f602e71d41eb" containerName="payload-processing" containerID="cri-o://58cf30e142f82bb78c569f08681a84a9918e4268d7f498c9c478083969ed5c2c" gracePeriod=30
Oct 30 21:55:34.525405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.525321    4549 generic.go:356] "Generic (PLEG): container finished" podID="a4111e34-86a0-4dd6-a4f7-f602e71d41eb" containerID="58cf30e142f82bb78c569f08681a84a9918e4268d7f498c9c478083969ed5c2c" exitCode=0
Oct 30 21:55:34.525405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.525380    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg" event={"ID":"a4111e34-86a0-4dd6-a4f7-f602e71d41eb","Type":"ContainerDied","Data":"58cf30e142f82bb78c569f08681a84a9918e4268d7f498c9c478083969ed5c2c"}
Oct 30 21:55:34.535125 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.535095    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.649919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.649886    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:55:34.696280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.695459    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.816579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.815461    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume\") pod \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") "
Oct 30 21:55:34.816579 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.815662    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk\" (UniqueName: \"kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk\") pod \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\" (UID: \"a4111e34-86a0-4dd6-a4f7-f602e71d41eb\") "
Oct 30 21:55:34.860209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.860156    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume" pod "a4111e34-86a0-4dd6-a4f7-f602e71d41eb" (UID: "a4111e34-86a0-4dd6-a4f7-f602e71d41eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:34.862062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.862020    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk" pod "a4111e34-86a0-4dd6-a4f7-f602e71d41eb" (UID: "a4111e34-86a0-4dd6-a4f7-f602e71d41eb"). InnerVolumeSpecName "kube-api-access-bqffk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:34.908856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.907816    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:34.918086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.918028    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqffk\" (UniqueName: \"kubernetes.io/projected/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-kube-api-access-bqffk\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:34.918086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.918060    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4111e34-86a0-4dd6-a4f7-f602e71d41eb-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:34.932800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:34.932772    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:35.094917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.089258    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:35.240637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.240597    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:35.345015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.344933    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" path="/var/lib/kubelet/pods/51eabe42-4d52-4337-ab89-418bc86f0849/volumes"
Oct 30 21:55:35.345524 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.345507    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" path="/var/lib/kubelet/pods/5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64/volumes"
Oct 30 21:55:35.533663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.533628    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"
Oct 30 21:55:35.534188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.533637    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg" event={"ID":"a4111e34-86a0-4dd6-a4f7-f602e71d41eb","Type":"ContainerDied","Data":"3489afc909ffd26797654c127d869889106e92c52fba5f819b76874e30c14984"}
Oct 30 21:55:35.534188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.533785    4549 scope.go:122] "RemoveContainer" containerID="58cf30e142f82bb78c569f08681a84a9918e4268d7f498c9c478083969ed5c2c"
Oct 30 21:55:35.596925 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.596806    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"]
Oct 30 21:55:35.608121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.608088    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg"]
Oct 30 21:55:35.663285 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:35.663252    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:36.305314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:36.305224    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:36.481415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:36.481376    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:36.989856 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:36.978450    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:37.344606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:37.344570    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4111e34-86a0-4dd6-a4f7-f602e71d41eb" path="/var/lib/kubelet/pods/a4111e34-86a0-4dd6-a4f7-f602e71d41eb/volumes"
Oct 30 21:55:38.273537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:38.273504    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.417854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:38.410862    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.595677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:38.595633    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"]
Oct 30 21:55:38.595917 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:38.595881    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy" containerID="cri-o://cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a" gracePeriod=30
Oct 30 21:55:38.641377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:38.641328    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.861042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:38.860950    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:40.571221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.571186    4549 generic.go:356] "Generic (PLEG): container finished" podID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerID="6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd" exitCode=0
Oct 30 21:55:40.571689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.571261    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" event={"ID":"e8adf648-d12f-4e04-95be-2b76723a4b97","Type":"ContainerDied","Data":"6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"}
Oct 30 21:55:40.571742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.571691    4549 scope.go:122] "RemoveContainer" containerID="6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"
Oct 30 21:55:40.573380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.573351    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:55:40.573454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.573420    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:55:40.573492 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.573454    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:55:40.797605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:40.797566    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:41.106428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:41.106356    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:41.588217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:41.588176    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" event={"ID":"e8adf648-d12f-4e04-95be-2b76723a4b97","Type":"ContainerStarted","Data":"478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd"}
Oct 30 21:55:41.655515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:41.655479    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:55:41.656742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:41.656716    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:55:41.656902 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:41.656772    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused"
Oct 30 21:55:41.920680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:41.920512    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:42.595229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:42.595173    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:42.655822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:42.655789    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:55:42.656023 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:42.655882    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused"
Oct 30 21:55:42.828580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:42.828531    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:43.046334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:43.046212    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:43.655676 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:43.655638    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:55:43.656204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:43.655704    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused"
Oct 30 21:55:43.813782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:43.813735    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:43.960700 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:43.960674    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:55:44.094337 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.094291    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.094697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.094667    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.094883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.094743    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:44.094883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.094751    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095018 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.094922    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095030    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095066    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095133    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095203 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095163    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095199    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs\") pod \"a1ced10c-7ba6-419e-ab0c-b062a870400d\" (UID: \"a1ced10c-7ba6-419e-ab0c-b062a870400d\") "
Oct 30 21:55:44.095844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095805    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ced10c-7ba6-419e-ab0c-b062a870400d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.095965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.095873    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:44.096096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.096067    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:44.096172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.096133    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:44.096240 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.096198    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:44.097821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.097798    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:44.101072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.101043    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:55:44.103485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.103019    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:44.103485 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.103210    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch" pod "a1ced10c-7ba6-419e-ab0c-b062a870400d" (UID: "a1ced10c-7ba6-419e-ab0c-b062a870400d"). InnerVolumeSpecName "kube-api-access-gkfch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197107    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197140    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197153    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197166    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197177    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197190    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkfch\" (UniqueName: \"kubernetes.io/projected/a1ced10c-7ba6-419e-ab0c-b062a870400d-kube-api-access-gkfch\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197202    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.197206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.197214    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ced10c-7ba6-419e-ab0c-b062a870400d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:44.625911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.625866    4549 generic.go:356] "Generic (PLEG): container finished" podID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerID="cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a" exitCode=0
Oct 30 21:55:44.626076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.625913    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"
Oct 30 21:55:44.626076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.625922    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" event={"ID":"a1ced10c-7ba6-419e-ab0c-b062a870400d","Type":"ContainerDied","Data":"cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a"}
Oct 30 21:55:44.626076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.625988    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh" event={"ID":"a1ced10c-7ba6-419e-ab0c-b062a870400d","Type":"ContainerDied","Data":"4d9d5274044aba9859932401e4400ce40a7a0fe251db171c16c0c3680648972a"}
Oct 30 21:55:44.626076 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.626019    4549 scope.go:122] "RemoveContainer" containerID="cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a"
Oct 30 21:55:44.637017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.636995    4549 scope.go:122] "RemoveContainer" containerID="ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"
Oct 30 21:55:44.646815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.646794    4549 scope.go:122] "RemoveContainer" containerID="cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a"
Oct 30 21:55:44.647127 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:44.647108    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a\": container with ID starting with cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a not found: ID does not exist" containerID="cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a"
Oct 30 21:55:44.647179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.647137    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a"} err="failed to get container status \"cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a\": rpc error: code = NotFound desc = could not find container \"cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a\": container with ID starting with cdcdbaf3ae1a0d129375848ad8e9e3be799ad6e9ff36caaac2dd485318a1791a not found: ID does not exist"
Oct 30 21:55:44.647179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.647157    4549 scope.go:122] "RemoveContainer" containerID="ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"
Oct 30 21:55:44.647437 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:44.647415    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b\": container with ID starting with ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b not found: ID does not exist" containerID="ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"
Oct 30 21:55:44.647497 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.647443    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b"} err="failed to get container status \"ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b\": rpc error: code = NotFound desc = could not find container \"ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b\": container with ID starting with ce4dfab95f572a547fe7ae006cc525aa5755d4b4c96fde9cea6dcc471a819b8b not found: ID does not exist"
Oct 30 21:55:44.656134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.656109    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:55:44.656438 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.656167    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused"
Oct 30 21:55:44.694019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.693982    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:55:44.713069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.713035    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"]
Oct 30 21:55:44.718754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.718719    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh"]
Oct 30 21:55:44.907396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:44.907299    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:45.344822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:45.344775    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" path="/var/lib/kubelet/pods/a1ced10c-7ba6-419e-ab0c-b062a870400d/volumes"
Oct 30 21:55:45.655696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:45.655610    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:55:45.655696 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:45.655673    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused"
Oct 30 21:55:46.124791 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:46.124746    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:46.382542 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:46.382449    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:46.648231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:46.648144    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:46.655144 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:46.655122    4549 patch_prober.go:29] interesting pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused" start-of-body=
Oct 30 21:55:46.655258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:46.655179    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.221:15021/healthz/ready\": dial tcp 10.133.0.221:15021: connect: connection refused"
Oct 30 21:55:47.661647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:47.661614    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:55:47.662821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:47.662804    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:55:47.673174 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:47.673140    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:47.831859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:47.826647    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"]
Oct 30 21:55:47.861010 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:47.860976    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"]
Oct 30 21:55:47.987394 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:47.987306    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:48.235058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:48.235025    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:48.427974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:48.427944    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:48.986479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:48.986426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:50.107310 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.107278    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:50.404183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.404155    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:55:50.493959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.493932    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume\") pod \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") "
Oct 30 21:55:50.494160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.494040    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2\" (UniqueName: \"kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2\") pod \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\" (UID: \"9c2a12c4-4541-45a6-b3ca-913cc45d0491\") "
Oct 30 21:55:50.494403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.494378    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume" pod "9c2a12c4-4541-45a6-b3ca-913cc45d0491" (UID: "9c2a12c4-4541-45a6-b3ca-913cc45d0491"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:50.494580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.494562    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c2a12c4-4541-45a6-b3ca-913cc45d0491-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:50.496665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.496638    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2" pod "9c2a12c4-4541-45a6-b3ca-913cc45d0491" (UID: "9c2a12c4-4541-45a6-b3ca-913cc45d0491"). InnerVolumeSpecName "kube-api-access-dm8p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:50.595591 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.595551    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm8p2\" (UniqueName: \"kubernetes.io/projected/9c2a12c4-4541-45a6-b3ca-913cc45d0491-kube-api-access-dm8p2\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:50.668027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.667938    4549 generic.go:356] "Generic (PLEG): container finished" podID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerID="361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc" exitCode=0
Oct 30 21:55:50.668027 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.667989    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"
Oct 30 21:55:50.668269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.668029    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" event={"ID":"9c2a12c4-4541-45a6-b3ca-913cc45d0491","Type":"ContainerDied","Data":"361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc"}
Oct 30 21:55:50.668269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.668089    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h" event={"ID":"9c2a12c4-4541-45a6-b3ca-913cc45d0491","Type":"ContainerDied","Data":"222237e26374ba1220355ec99bdac9a7134fd57105cc6411bca7ce8e5f834d9f"}
Oct 30 21:55:50.668269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.668112    4549 scope.go:122] "RemoveContainer" containerID="361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc"
Oct 30 21:55:50.681446 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.681427    4549 scope.go:122] "RemoveContainer" containerID="361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc"
Oct 30 21:55:50.681736 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:50.681716    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc\": container with ID starting with 361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc not found: ID does not exist" containerID="361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc"
Oct 30 21:55:50.681810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.681748    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc"} err="failed to get container status \"361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc\": rpc error: code = NotFound desc = could not find container \"361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc\": container with ID starting with 361b44a7e1f9a20c9cdb0886be03e1207119e3f0eced64aee6cb068995a2d4cc not found: ID does not exist"
Oct 30 21:55:50.730742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.730710    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"]
Oct 30 21:55:50.742467 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:50.742435    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h"]
Oct 30 21:55:51.344156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:51.344122    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" path="/var/lib/kubelet/pods/9c2a12c4-4541-45a6-b3ca-913cc45d0491/volumes"
Oct 30 21:55:51.787702 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:51.787595    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:52.828118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:52.828071    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq" podUID="e354e8ef-2a80-468c-874f-255626055aa1" containerName="payload-processing" containerID="cri-o://04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a" gracePeriod=25
Oct 30 21:55:52.861617 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:52.861583    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq" podUID="2cc788fd-f20b-45b4-aac3-6c50a7f3f367" containerName="payload-pre-processing" containerID="cri-o://51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53" gracePeriod=25
Oct 30 21:55:53.137097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.137055    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:55:53.203967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.203939    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:55:53.227080 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.227052    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume\") pod \"e354e8ef-2a80-468c-874f-255626055aa1\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") "
Oct 30 21:55:53.227267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.227087    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t\" (UniqueName: \"kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t\") pod \"e354e8ef-2a80-468c-874f-255626055aa1\" (UID: \"e354e8ef-2a80-468c-874f-255626055aa1\") "
Oct 30 21:55:53.227427 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.227400    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume" pod "e354e8ef-2a80-468c-874f-255626055aa1" (UID: "e354e8ef-2a80-468c-874f-255626055aa1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:53.229580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.229560    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t" pod "e354e8ef-2a80-468c-874f-255626055aa1" (UID: "e354e8ef-2a80-468c-874f-255626055aa1"). InnerVolumeSpecName "kube-api-access-s8l9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:53.266636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.266604    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:53.328207 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.328170    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b\") pod \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") "
Oct 30 21:55:53.328396 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.328283    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume\") pod \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\" (UID: \"2cc788fd-f20b-45b4-aac3-6c50a7f3f367\") "
Oct 30 21:55:53.328654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.328641    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e354e8ef-2a80-468c-874f-255626055aa1-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:53.328710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.328656    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8l9t\" (UniqueName: \"kubernetes.io/projected/e354e8ef-2a80-468c-874f-255626055aa1-kube-api-access-s8l9t\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:53.328710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.328664    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume" pod "2cc788fd-f20b-45b4-aac3-6c50a7f3f367" (UID: "2cc788fd-f20b-45b4-aac3-6c50a7f3f367"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:53.330671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.330644    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b" pod "2cc788fd-f20b-45b4-aac3-6c50a7f3f367" (UID: "2cc788fd-f20b-45b4-aac3-6c50a7f3f367"). InnerVolumeSpecName "kube-api-access-rqx7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:53.430855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.430150    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-kube-api-access-rqx7b\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:53.430855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.430192    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cc788fd-f20b-45b4-aac3-6c50a7f3f367-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:53.700216 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.700133    4549 generic.go:356] "Generic (PLEG): container finished" podID="2cc788fd-f20b-45b4-aac3-6c50a7f3f367" containerID="51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53" exitCode=0
Oct 30 21:55:53.700630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.700598    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq" event={"ID":"2cc788fd-f20b-45b4-aac3-6c50a7f3f367","Type":"ContainerDied","Data":"51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53"}
Oct 30 21:55:53.700890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.700816    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq" event={"ID":"2cc788fd-f20b-45b4-aac3-6c50a7f3f367","Type":"ContainerDied","Data":"c7e7d4aba09588322cb47db01e9e74b422f0cc7e2096fbdc5ffaedc3b0e3cd40"}
Oct 30 21:55:53.701001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.700897    4549 scope.go:122] "RemoveContainer" containerID="51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53"
Oct 30 21:55:53.701066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.701055    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"
Oct 30 21:55:53.703935 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.703891    4549 generic.go:356] "Generic (PLEG): container finished" podID="e354e8ef-2a80-468c-874f-255626055aa1" containerID="04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a" exitCode=0
Oct 30 21:55:53.704090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.704071    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq" event={"ID":"e354e8ef-2a80-468c-874f-255626055aa1","Type":"ContainerDied","Data":"04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a"}
Oct 30 21:55:53.704139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.704125    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq" event={"ID":"e354e8ef-2a80-468c-874f-255626055aa1","Type":"ContainerDied","Data":"c7473de77c0a30ffd2b4529ef2b139ae7696e75cbe87f7718b7b6c3a2711501f"}
Oct 30 21:55:53.704222 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.704208    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"
Oct 30 21:55:53.714094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.714064    4549 scope.go:122] "RemoveContainer" containerID="51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53"
Oct 30 21:55:53.714497 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:53.714475    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53\": container with ID starting with 51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53 not found: ID does not exist" containerID="51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53"
Oct 30 21:55:53.714606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.714510    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53"} err="failed to get container status \"51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53\": rpc error: code = NotFound desc = could not find container \"51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53\": container with ID starting with 51d369bf7e70dec064b63a08be46d000b5ae5a31b27b91cbf71084bf5cdb1e53 not found: ID does not exist"
Oct 30 21:55:53.714606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.714534    4549 scope.go:122] "RemoveContainer" containerID="04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a"
Oct 30 21:55:53.724449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.724428    4549 scope.go:122] "RemoveContainer" containerID="04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a"
Oct 30 21:55:53.724810 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:53.724753    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a\": container with ID starting with 04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a not found: ID does not exist" containerID="04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a"
Oct 30 21:55:53.724810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.724795    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a"} err="failed to get container status \"04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a\": rpc error: code = NotFound desc = could not find container \"04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a\": container with ID starting with 04ecb6a61ba8f9370d2973dcd60cfdca5ee2a64c42147c42a6f222132702106a not found: ID does not exist"
Oct 30 21:55:53.760724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.760670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:53.761674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.761647    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"]
Oct 30 21:55:53.767416 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.767390    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq"]
Oct 30 21:55:53.800068 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.800026    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"]
Oct 30 21:55:53.814776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.814743    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq"]
Oct 30 21:55:53.958193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:53.958107    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:54.366269 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:54.366232    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:54.635199 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:54.635117    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"]
Oct 30 21:55:54.698228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:54.698192    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"]
Oct 30 21:55:54.816104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:54.816072    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:54.998921 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:54.998841    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:55.345647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:55.345610    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cc788fd-f20b-45b4-aac3-6c50a7f3f367" path="/var/lib/kubelet/pods/2cc788fd-f20b-45b4-aac3-6c50a7f3f367/volumes"
Oct 30 21:55:55.346191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:55.346169    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e354e8ef-2a80-468c-874f-255626055aa1" path="/var/lib/kubelet/pods/e354e8ef-2a80-468c-874f-255626055aa1/volumes"
Oct 30 21:55:55.587013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:55.586971    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:56.056719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:56.056668    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:56.208887 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:56.208845    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:56.545857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:56.545733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:56.583587 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:56.583546    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:55:57.080776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:57.080736    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:57.727369 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:57.727326    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"]
Oct 30 21:55:57.727650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:57.727602    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn" podUID="96cd0105-e983-441f-80c2-55572a827e1a" containerName="maas-api" containerID="cri-o://f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc" gracePeriod=30
Oct 30 21:55:57.832354 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:57.832285    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:58.039180 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.039150    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:55:58.108432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.108389    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs\") pod \"96cd0105-e983-441f-80c2-55572a827e1a\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") "
Oct 30 21:55:58.109933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.109894    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls\") pod \"96cd0105-e983-441f-80c2-55572a827e1a\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") "
Oct 30 21:55:58.152507 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.152479    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca\") pod \"96cd0105-e983-441f-80c2-55572a827e1a\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") "
Oct 30 21:55:58.155202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.155180    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg\" (UniqueName: \"kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg\") pod \"96cd0105-e983-441f-80c2-55572a827e1a\" (UID: \"96cd0105-e983-441f-80c2-55572a827e1a\") "
Oct 30 21:55:58.190051 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.155009    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca" pod "96cd0105-e983-441f-80c2-55572a827e1a" (UID: "96cd0105-e983-441f-80c2-55572a827e1a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:58.190258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.165286    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs" pod "96cd0105-e983-441f-80c2-55572a827e1a" (UID: "96cd0105-e983-441f-80c2-55572a827e1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.190258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.172481    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg" pod "96cd0105-e983-441f-80c2-55572a827e1a" (UID: "96cd0105-e983-441f-80c2-55572a827e1a"). InnerVolumeSpecName "kube-api-access-jk2mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:58.190258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.173010    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls" pod "96cd0105-e983-441f-80c2-55572a827e1a" (UID: "96cd0105-e983-441f-80c2-55572a827e1a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.190569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.190535    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:58.190618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.190574    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96cd0105-e983-441f-80c2-55572a827e1a-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:58.190618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.190591    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96cd0105-e983-441f-80c2-55572a827e1a-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:58.190618 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.190605    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk2mg\" (UniqueName: \"kubernetes.io/projected/96cd0105-e983-441f-80c2-55572a827e1a-kube-api-access-jk2mg\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:55:58.389658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.389363    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:58.574573 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.574533    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:58.740508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.740422    4549 generic.go:356] "Generic (PLEG): container finished" podID="96cd0105-e983-441f-80c2-55572a827e1a" containerID="f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc" exitCode=0
Oct 30 21:55:58.740508 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.740482    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"
Oct 30 21:55:58.740724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.740516    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn" event={"ID":"96cd0105-e983-441f-80c2-55572a827e1a","Type":"ContainerDied","Data":"f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc"}
Oct 30 21:55:58.740724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.740573    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn" event={"ID":"96cd0105-e983-441f-80c2-55572a827e1a","Type":"ContainerDied","Data":"65ef2c474628bfabaddc289687a2c247791d716ab66d6764202eb1e74c961ab2"}
Oct 30 21:55:58.740724 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.740595    4549 scope.go:122] "RemoveContainer" containerID="f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc"
Oct 30 21:55:58.759066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.759042    4549 scope.go:122] "RemoveContainer" containerID="f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc"
Oct 30 21:55:58.759388 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:55:58.759367    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc\": container with ID starting with f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc not found: ID does not exist" containerID="f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc"
Oct 30 21:55:58.759443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.759398    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc"} err="failed to get container status \"f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc\": rpc error: code = NotFound desc = could not find container \"f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc\": container with ID starting with f78bdde5c6fc2487890b4f6a8ffed04f9ddbcb6ff953ad97fbc067c7d66f02fc not found: ID does not exist"
Oct 30 21:55:58.799386 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.799358    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"]
Oct 30 21:55:58.807126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:58.807089    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-79477e-649557d5ff-jt9xn"]
Oct 30 21:55:59.009402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:59.009309    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:59.343845 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:59.343801    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96cd0105-e983-441f-80c2-55572a827e1a" path="/var/lib/kubelet/pods/96cd0105-e983-441f-80c2-55572a827e1a/volumes"
Oct 30 21:55:59.635775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:59.635668    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" containerName="payload-processing" containerID="cri-o://e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173" gracePeriod=25
Oct 30 21:55:59.698904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:55:59.698839    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerName="payload-pre-processing" containerID="cri-o://02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227" gracePeriod=25
Oct 30 21:56:00.319089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:00.319043    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:01.891261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:01.891228    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"]
Oct 30 21:56:02.183560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:02.183453    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:02.424449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:02.424409    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:04.261019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:04.260981    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:04.577651 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:04.577620    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:04.690443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:04.690405    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"]
Oct 30 21:56:04.690710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:04.690682    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy" containerID="cri-o://aeb7fec8f8853dde6cb299414698f7e3a10daf323f3dfda66256bdad6bfccd10" gracePeriod=30
Oct 30 21:56:04.907286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:04.907198    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:05.102708 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:05.102666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:05.474377 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:05.474329    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:06.389103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:06.389055    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:07.826342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:07.826308    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"]
Oct 30 21:56:07.964284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:07.964243    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:08.826129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:08.826088    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:08.826343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:08.826289    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:09.840665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:09.840630    4549 generic.go:356] "Generic (PLEG): container finished" podID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerID="aeb7fec8f8853dde6cb299414698f7e3a10daf323f3dfda66256bdad6bfccd10" exitCode=0
Oct 30 21:56:09.841197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:09.840704    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" event={"ID":"2c5bc90d-b046-4a1f-88de-93bbb58aac27","Type":"ContainerDied","Data":"aeb7fec8f8853dde6cb299414698f7e3a10daf323f3dfda66256bdad6bfccd10"}
Oct 30 21:56:09.841197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:09.840772    4549 scope.go:122] "RemoveContainer" containerID="e6981a78c4ffd977f09c2c5f95ba7e73bbe9767e37e9067f56f4ff6ee33bbd30"
Oct 30 21:56:09.881730 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:09.881695    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"]
Oct 30 21:56:09.882038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:09.881979    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" podUID="8b350545-9354-45da-9481-8012e1f8d398" containerName="maas-api" containerID="cri-o://5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611" gracePeriod=30
Oct 30 21:56:09.965733 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:09.965594    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" podUID="8b350545-9354-45da-9481-8012e1f8d398" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.99:8443/health\": dial tcp 10.133.0.99:8443: connect: connection refused"
Oct 30 21:56:10.018443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.018416    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:56:10.048405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.048371    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:10.134502 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.134475    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:56:10.185914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.185876    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.186272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.186250    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.186495 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.186474    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.186695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.186669    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.186933 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.186916    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.187092 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.187075    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.187274 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.187261    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.187412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.187398    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.187528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.187515    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert\") pod \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\" (UID: \"2c5bc90d-b046-4a1f-88de-93bbb58aac27\") "
Oct 30 21:56:10.188774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.188749    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:10.190074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.190033    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:10.190251 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.190232    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:10.190434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.190416    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:10.190647 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.190604    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:10.192711 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.192660    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:10.193286 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.193259    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:10.193863 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.193806    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:56:10.194206 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.194182    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf" pod "2c5bc90d-b046-4a1f-88de-93bbb58aac27" (UID: "2c5bc90d-b046-4a1f-88de-93bbb58aac27"). InnerVolumeSpecName "kube-api-access-khqhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:10.209035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.209001    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"]
Oct 30 21:56:10.289258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.289172    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls\") pod \"8b350545-9354-45da-9481-8012e1f8d398\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") "
Oct 30 21:56:10.289258 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.289217    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca\") pod \"8b350545-9354-45da-9481-8012e1f8d398\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") "
Oct 30 21:56:10.289468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.289279    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm\" (UniqueName: \"kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm\") pod \"8b350545-9354-45da-9481-8012e1f8d398\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") "
Oct 30 21:56:10.289468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.289434    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs\") pod \"8b350545-9354-45da-9481-8012e1f8d398\" (UID: \"8b350545-9354-45da-9481-8012e1f8d398\") "
Oct 30 21:56:10.289686 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.289657    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca" pod "8b350545-9354-45da-9481-8012e1f8d398" (UID: "8b350545-9354-45da-9481-8012e1f8d398"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:10.290259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290232    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290260    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290275    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290290    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290305    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290317    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5bc90d-b046-4a1f-88de-93bbb58aac27-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290331    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290344    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5bc90d-b046-4a1f-88de-93bbb58aac27-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290357    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b350545-9354-45da-9481-8012e1f8d398-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.290454 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.290373    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khqhf\" (UniqueName: \"kubernetes.io/projected/2c5bc90d-b046-4a1f-88de-93bbb58aac27-kube-api-access-khqhf\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.291864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.291804    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs" pod "8b350545-9354-45da-9481-8012e1f8d398" (UID: "8b350545-9354-45da-9481-8012e1f8d398"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:10.291953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.291866    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls" pod "8b350545-9354-45da-9481-8012e1f8d398" (UID: "8b350545-9354-45da-9481-8012e1f8d398"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:10.292019 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.292001    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm" pod "8b350545-9354-45da-9481-8012e1f8d398" (UID: "8b350545-9354-45da-9481-8012e1f8d398"). InnerVolumeSpecName "kube-api-access-htqgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:10.397817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.397775    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.397817 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.397819    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b350545-9354-45da-9481-8012e1f8d398-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.398111 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.397855    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htqgm\" (UniqueName: \"kubernetes.io/projected/8b350545-9354-45da-9481-8012e1f8d398-kube-api-access-htqgm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:10.460293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.460261    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:10.704462 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.704427    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:10.862951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.862912    4549 generic.go:356] "Generic (PLEG): container finished" podID="8b350545-9354-45da-9481-8012e1f8d398" containerID="5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611" exitCode=0
Oct 30 21:56:10.863447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.862971    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"
Oct 30 21:56:10.863447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.862998    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" event={"ID":"8b350545-9354-45da-9481-8012e1f8d398","Type":"ContainerDied","Data":"5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611"}
Oct 30 21:56:10.863447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.863063    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4" event={"ID":"8b350545-9354-45da-9481-8012e1f8d398","Type":"ContainerDied","Data":"4c1023c11d0b8bfcef5f2b502ea5910eeeb0d77d43e2162e5e831917f1f0d421"}
Oct 30 21:56:10.863447 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.863104    4549 scope.go:122] "RemoveContainer" containerID="5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611"
Oct 30 21:56:10.865090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.865072    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"
Oct 30 21:56:10.865234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.865113    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh" event={"ID":"2c5bc90d-b046-4a1f-88de-93bbb58aac27","Type":"ContainerDied","Data":"ed1e45e51e953a86b3228a7f725e26cc73bfc413fd72bcff36231ba5bf9e1298"}
Oct 30 21:56:10.876204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.876165    4549 scope.go:122] "RemoveContainer" containerID="5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611"
Oct 30 21:56:10.876627 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:10.876549    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611\": container with ID starting with 5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611 not found: ID does not exist" containerID="5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611"
Oct 30 21:56:10.876703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.876639    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611"} err="failed to get container status \"5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611\": rpc error: code = NotFound desc = could not find container \"5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611\": container with ID starting with 5253e138e279f012c83b95a7120444cdba15fc6d34c9a78393d483b8575f1611 not found: ID does not exist"
Oct 30 21:56:10.876703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.876658    4549 scope.go:122] "RemoveContainer" containerID="aeb7fec8f8853dde6cb299414698f7e3a10daf323f3dfda66256bdad6bfccd10"
Oct 30 21:56:10.944040 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.943994    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"]
Oct 30 21:56:10.966158 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:10.966065    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh"]
Oct 30 21:56:11.028434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:11.028399    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"]
Oct 30 21:56:11.041710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:11.041673    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-e11bef-547674bcd-54px4"]
Oct 30 21:56:11.178993 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:11.178940    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:11.344311 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:11.344281    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" path="/var/lib/kubelet/pods/2c5bc90d-b046-4a1f-88de-93bbb58aac27/volumes"
Oct 30 21:56:11.344746 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:11.344732    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b350545-9354-45da-9481-8012e1f8d398" path="/var/lib/kubelet/pods/8b350545-9354-45da-9481-8012e1f8d398/volumes"
Oct 30 21:56:11.415073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:11.415032    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.552766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.552724    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.756361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.756326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.830854 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.828437    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerName="payload-pre-processing" containerID="cri-o://3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25" gracePeriod=25
Oct 30 21:56:12.886276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886244    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"]
Oct 30 21:56:12.886815 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886802    4549 cpu_manager.go:414] "containerMap: removing container" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886816    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886850    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886858    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886869    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e354e8ef-2a80-468c-874f-255626055aa1" containerName="payload-processing"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886877    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e354e8ef-2a80-468c-874f-255626055aa1" containerName="payload-processing"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886887    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy"
Oct 30 21:56:12.886905 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886894    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886913    4549 cpu_manager.go:414] "containerMap: removing container" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886919    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886926    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886930    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886946    4549 cpu_manager.go:414] "containerMap: removing container" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886951    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886960    4549 cpu_manager.go:414] "containerMap: removing container" podUID="bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886965    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886972    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2cc788fd-f20b-45b4-aac3-6c50a7f3f367" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886977    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cc788fd-f20b-45b4-aac3-6c50a7f3f367" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886988    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.886992    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887001    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a4111e34-86a0-4dd6-a4f7-f602e71d41eb" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887005    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4111e34-86a0-4dd6-a4f7-f602e71d41eb" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887013    4549 cpu_manager.go:414] "containerMap: removing container" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887018    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887027    4549 cpu_manager.go:414] "containerMap: removing container" podUID="28fb0f05-db53-49e3-8d70-9aa2df134292" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887031    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28fb0f05-db53-49e3-8d70-9aa2df134292" containerName="payload-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887039    4549 cpu_manager.go:414] "containerMap: removing container" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887044    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" containerName="payload-pre-processing"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887051    4549 cpu_manager.go:414] "containerMap: removing container" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887055    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887062    4549 cpu_manager.go:414] "containerMap: removing container" podUID="8b350545-9354-45da-9481-8012e1f8d398" containerName="maas-api"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887067    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b350545-9354-45da-9481-8012e1f8d398" containerName="maas-api"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887073    4549 cpu_manager.go:414] "containerMap: removing container" podUID="96cd0105-e983-441f-80c2-55572a827e1a" containerName="maas-api"
Oct 30 21:56:12.887137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887077    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96cd0105-e983-441f-80c2-55572a827e1a" containerName="maas-api"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887164    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfc8fe6a-8b7f-4b78-ab0e-05d7d6b53fda" containerName="payload-pre-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887173    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cc788fd-f20b-45b4-aac3-6c50a7f3f367" containerName="payload-pre-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887181    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887188    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4111e34-86a0-4dd6-a4f7-f602e71d41eb" containerName="payload-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887194    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c2a12c4-4541-45a6-b3ca-913cc45d0491" containerName="payload-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887201    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="28fb0f05-db53-49e3-8d70-9aa2df134292" containerName="payload-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887207    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="51eabe42-4d52-4337-ab89-418bc86f0849" containerName="payload-pre-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887213    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887218    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b350545-9354-45da-9481-8012e1f8d398" containerName="maas-api"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887222    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="96cd0105-e983-441f-80c2-55572a827e1a" containerName="maas-api"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887229    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="74aba30e-2e9d-4dc1-a75b-36aa0e70bc6e" containerName="istio-proxy"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887235    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e354e8ef-2a80-468c-874f-255626055aa1" containerName="payload-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887241    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c5bc90d-b046-4a1f-88de-93bbb58aac27" containerName="istio-proxy"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887247    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a9bd599-6ee1-4572-ae7c-f36f7d0d4b64" containerName="payload-processing"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887252    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff1362cb-2c95-45a4-8a59-6c7a2d335192" containerName="main"
Oct 30 21:56:12.887924 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.887259    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c405a84-a806-4b83-a9ed-6b2eafd74fb8" containerName="payload-pre-processing"
Oct 30 21:56:12.892081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.892060    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:12.912914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.912886    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"]
Oct 30 21:56:12.921212 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.921167    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-480d41\""
Oct 30 21:56:12.921384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:12.921363    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-480d41-dockercfg-lkhb7\""
Oct 30 21:56:13.015970 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.015934    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"]
Oct 30 21:56:13.016658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.016643    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1ced10c-7ba6-419e-ab0c-b062a870400d" containerName="istio-proxy"
Oct 30 21:56:13.020053 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.020038    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.030417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.030392    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.030540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.030498    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7zql\" (UniqueName: \"kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql\") pod \"payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.030592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.030577    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5n2c\" (UniqueName: \"kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c\") pod \"payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.030636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.030612    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.043606 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.043580    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"]
Oct 30 21:56:13.131398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.131306    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.131398 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.131381    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7zql\" (UniqueName: \"kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql\") pod \"payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.131596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.131422    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5n2c\" (UniqueName: \"kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c\") pod \"payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.131596 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.131444    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.132152 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.132131    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.132223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.132176    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.155800 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.155768    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7zql\" (UniqueName: \"kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql\") pod \"payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.155953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.155858    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5n2c\" (UniqueName: \"kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c\") pod \"payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.204979 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.204943    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:13.333597 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.333557    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.459585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.459544    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:13.534865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.534792    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"]
Oct 30 21:56:13.539267 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:56:13.539227    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f51833_a02d_40c7_8067_7364fca1aff2.slice/crio-7f4442e562f733d401d7f94afe8e9fd07729c6b4789e7af20514dbfcfbbff059 WatchSource:0}: Error finding container 7f4442e562f733d401d7f94afe8e9fd07729c6b4789e7af20514dbfcfbbff059: Status 404 returned error can't find the container with id 7f4442e562f733d401d7f94afe8e9fd07729c6b4789e7af20514dbfcfbbff059
Oct 30 21:56:13.549146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.549115    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"]
Oct 30 21:56:13.549362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.549329    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54" podUID="67acd37b-ca60-4237-bfd6-bb38757d71bc" containerName="payload-processing" containerID="cri-o://131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef" gracePeriod=30
Oct 30 21:56:13.584356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.584320    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"]
Oct 30 21:56:13.584816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.584540    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn" podUID="b5898524-e326-4e39-bdce-3aaff9b2b5b9" containerName="payload-pre-processing" containerID="cri-o://e06d672d79ba06ad29f0cf7556aaae09c7f6002f4e79acdd7202636092476f07" gracePeriod=30
Oct 30 21:56:13.673374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.673348    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"]
Oct 30 21:56:13.674937 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:56:13.674909    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1482a83_e8a0_40a9_86de_755c84032fc4.slice/crio-9f7648b1886a176ec95277ff698572fa2fcfc5131afd0da7d250c720bb7d156d WatchSource:0}: Error finding container 9f7648b1886a176ec95277ff698572fa2fcfc5131afd0da7d250c720bb7d156d: Status 404 returned error can't find the container with id 9f7648b1886a176ec95277ff698572fa2fcfc5131afd0da7d250c720bb7d156d
Oct 30 21:56:13.687090 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.687060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:13.869327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.869295    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:56:13.947908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.944931    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z" event={"ID":"d1482a83-e8a0-40a9-86de-755c84032fc4","Type":"ContainerStarted","Data":"f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490"}
Oct 30 21:56:13.947908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.944997    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z" event={"ID":"d1482a83-e8a0-40a9-86de-755c84032fc4","Type":"ContainerStarted","Data":"9f7648b1886a176ec95277ff698572fa2fcfc5131afd0da7d250c720bb7d156d"}
Oct 30 21:56:13.947908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.945958    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:13.973984 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.972966    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.988250    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4" event={"ID":"f0f51833-a02d-40c7-8067-7364fca1aff2","Type":"ContainerStarted","Data":"cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3"}
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.988322    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4" event={"ID":"f0f51833-a02d-40c7-8067-7364fca1aff2","Type":"ContainerStarted","Data":"7f4442e562f733d401d7f94afe8e9fd07729c6b4789e7af20514dbfcfbbff059"}
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.997682    4549 generic.go:356] "Generic (PLEG): container finished" podID="b5898524-e326-4e39-bdce-3aaff9b2b5b9" containerID="e06d672d79ba06ad29f0cf7556aaae09c7f6002f4e79acdd7202636092476f07" exitCode=0
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.989112    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.998378    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn" event={"ID":"b5898524-e326-4e39-bdce-3aaff9b2b5b9","Type":"ContainerDied","Data":"e06d672d79ba06ad29f0cf7556aaae09c7f6002f4e79acdd7202636092476f07"}
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.999907    4549 generic.go:356] "Generic (PLEG): container finished" podID="67acd37b-ca60-4237-bfd6-bb38757d71bc" containerID="131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef" exitCode=0
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:13.999961    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54" event={"ID":"67acd37b-ca60-4237-bfd6-bb38757d71bc","Type":"ContainerDied","Data":"131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef"}
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.000001    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54" event={"ID":"67acd37b-ca60-4237-bfd6-bb38757d71bc","Type":"ContainerDied","Data":"4f5202f20d82343a207c37b1164cea76f1ff48410c3e05c552f726f57a27f229"}
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.000032    4549 scope.go:122] "RemoveContainer" containerID="131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef"
Oct 30 21:56:14.001309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.000190    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"
Oct 30 21:56:14.010009 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.009901    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:56:14.011560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.011518    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z" podStartSLOduration=2.011505704 podStartE2EDuration="2.011505704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:14.010439453 +0000 UTC m=+3356.784839383" watchObservedRunningTime="2026-10-30 21:56:14.011505704 +0000 UTC m=+3356.785905642"
Oct 30 21:56:14.019103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.019084    4549 scope.go:122] "RemoveContainer" containerID="131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef"
Oct 30 21:56:14.022055 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:14.021975    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef\": container with ID starting with 131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef not found: ID does not exist" containerID="131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef"
Oct 30 21:56:14.022055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.022024    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef"} err="failed to get container status \"131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef\": rpc error: code = NotFound desc = could not find container \"131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef\": container with ID starting with 131a77cd03c2686c4d22fbaff6092c1fb6fc7e1b22275aa0484bd658e010b9ef not found: ID does not exist"
Oct 30 21:56:14.045380 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.045347    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume\") pod \"67acd37b-ca60-4237-bfd6-bb38757d71bc\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") "
Oct 30 21:56:14.045578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.045426    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv\" (UniqueName: \"kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv\") pod \"67acd37b-ca60-4237-bfd6-bb38757d71bc\" (UID: \"67acd37b-ca60-4237-bfd6-bb38757d71bc\") "
Oct 30 21:56:14.045842 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.045798    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume" pod "67acd37b-ca60-4237-bfd6-bb38757d71bc" (UID: "67acd37b-ca60-4237-bfd6-bb38757d71bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:14.047802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.047779    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv" pod "67acd37b-ca60-4237-bfd6-bb38757d71bc" (UID: "67acd37b-ca60-4237-bfd6-bb38757d71bc"). InnerVolumeSpecName "kube-api-access-cf2zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:14.122133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.122077    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4" podStartSLOduration=2.122061899 podStartE2EDuration="2.122061899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:14.120128606 +0000 UTC m=+3356.894528546" watchObservedRunningTime="2026-10-30 21:56:14.122061899 +0000 UTC m=+3356.896461892"
Oct 30 21:56:14.147105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.147075    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume\") pod \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") "
Oct 30 21:56:14.147302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.147127    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6\" (UniqueName: \"kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6\") pod \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\" (UID: \"b5898524-e326-4e39-bdce-3aaff9b2b5b9\") "
Oct 30 21:56:14.147545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.147519    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume" pod "b5898524-e326-4e39-bdce-3aaff9b2b5b9" (UID: "b5898524-e326-4e39-bdce-3aaff9b2b5b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:14.147675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.147610    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67acd37b-ca60-4237-bfd6-bb38757d71bc-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:14.147675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.147635    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf2zv\" (UniqueName: \"kubernetes.io/projected/67acd37b-ca60-4237-bfd6-bb38757d71bc-kube-api-access-cf2zv\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:14.149738 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.149712    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6" pod "b5898524-e326-4e39-bdce-3aaff9b2b5b9" (UID: "b5898524-e326-4e39-bdce-3aaff9b2b5b9"). InnerVolumeSpecName "kube-api-access-p24j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:14.238187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.238081    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:14.248780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.248748    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5898524-e326-4e39-bdce-3aaff9b2b5b9-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:14.248780 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.248775    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p24j6\" (UniqueName: \"kubernetes.io/projected/b5898524-e326-4e39-bdce-3aaff9b2b5b9-kube-api-access-p24j6\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:14.356293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.356257    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"]
Oct 30 21:56:14.382681 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.378815    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54"]
Oct 30 21:56:14.411864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.404446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:14.735636 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.735595    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:14.911958 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:14.911920    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:15.008525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.008440    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"
Oct 30 21:56:15.008525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.008441    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn" event={"ID":"b5898524-e326-4e39-bdce-3aaff9b2b5b9","Type":"ContainerDied","Data":"55bb30b94c09438b515046782796e02083c8eb1f66f623df42a66c3c5a1c088e"}
Oct 30 21:56:15.008773 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.008570    4549 scope.go:122] "RemoveContainer" containerID="e06d672d79ba06ad29f0cf7556aaae09c7f6002f4e79acdd7202636092476f07"
Oct 30 21:56:15.075356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.075320    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"]
Oct 30 21:56:15.078859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.078817    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn"]
Oct 30 21:56:15.345187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.345153    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67acd37b-ca60-4237-bfd6-bb38757d71bc" path="/var/lib/kubelet/pods/67acd37b-ca60-4237-bfd6-bb38757d71bc/volumes"
Oct 30 21:56:15.345641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.345626    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5898524-e326-4e39-bdce-3aaff9b2b5b9" path="/var/lib/kubelet/pods/b5898524-e326-4e39-bdce-3aaff9b2b5b9/volumes"
Oct 30 21:56:15.993486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:15.993446    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"]
Oct 30 21:56:16.315287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:16.315252    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:16.492171 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:16.492137    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:16.913363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:16.913320    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main" containerID="cri-o://04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d" gracePeriod=45
Oct 30 21:56:17.178349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.178285    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:56:17.282493 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282454    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home\") pod \"b9ee38ff-372c-424c-892b-a3b4880e0c34\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") "
Oct 30 21:56:17.282682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282565    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f\" (UniqueName: \"kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f\") pod \"b9ee38ff-372c-424c-892b-a3b4880e0c34\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") "
Oct 30 21:56:17.282682 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282596    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache\") pod \"b9ee38ff-372c-424c-892b-a3b4880e0c34\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") "
Oct 30 21:56:17.282822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282723    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir\") pod \"b9ee38ff-372c-424c-892b-a3b4880e0c34\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") "
Oct 30 21:56:17.282822 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282801    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs\") pod \"b9ee38ff-372c-424c-892b-a3b4880e0c34\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") "
Oct 30 21:56:17.282985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282823    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home" pod "b9ee38ff-372c-424c-892b-a3b4880e0c34" (UID: "b9ee38ff-372c-424c-892b-a3b4880e0c34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:17.282985 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.282884    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm\") pod \"b9ee38ff-372c-424c-892b-a3b4880e0c34\" (UID: \"b9ee38ff-372c-424c-892b-a3b4880e0c34\") "
Oct 30 21:56:17.283117 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.283088    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir" pod "b9ee38ff-372c-424c-892b-a3b4880e0c34" (UID: "b9ee38ff-372c-424c-892b-a3b4880e0c34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:17.283388 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.283364    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:17.283513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.283392    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:17.283842 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.283802    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache" pod "b9ee38ff-372c-424c-892b-a3b4880e0c34" (UID: "b9ee38ff-372c-424c-892b-a3b4880e0c34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:17.285517 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.285493    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm" pod "b9ee38ff-372c-424c-892b-a3b4880e0c34" (UID: "b9ee38ff-372c-424c-892b-a3b4880e0c34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:17.285734 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.285709    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs" pod "b9ee38ff-372c-424c-892b-a3b4880e0c34" (UID: "b9ee38ff-372c-424c-892b-a3b4880e0c34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:17.287328 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.287305    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f" pod "b9ee38ff-372c-424c-892b-a3b4880e0c34" (UID: "b9ee38ff-372c-424c-892b-a3b4880e0c34"). InnerVolumeSpecName "kube-api-access-bvc9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:17.385059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.385024    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ee38ff-372c-424c-892b-a3b4880e0c34-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:17.385059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.385060    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:17.385302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.385075    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvc9f\" (UniqueName: \"kubernetes.io/projected/b9ee38ff-372c-424c-892b-a3b4880e0c34-kube-api-access-bvc9f\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:17.385302 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:17.385092    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ee38ff-372c-424c-892b-a3b4880e0c34-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:18.036441 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.036405    4549 generic.go:356] "Generic (PLEG): container finished" podID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerID="04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d" exitCode=0
Oct 30 21:56:18.036665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.036445    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"
Oct 30 21:56:18.036665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.036480    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" event={"ID":"b9ee38ff-372c-424c-892b-a3b4880e0c34","Type":"ContainerDied","Data":"04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d"}
Oct 30 21:56:18.036665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.036521    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l" event={"ID":"b9ee38ff-372c-424c-892b-a3b4880e0c34","Type":"ContainerDied","Data":"3cf78f527f9971f4285dd182e40c290bda6200918c89355d3a5d4a753839e658"}
Oct 30 21:56:18.036665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.036543    4549 scope.go:122] "RemoveContainer" containerID="04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d"
Oct 30 21:56:18.046267 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.046245    4549 scope.go:122] "RemoveContainer" containerID="04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d"
Oct 30 21:56:18.046631 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:18.046606    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d\": container with ID starting with 04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d not found: ID does not exist" containerID="04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d"
Oct 30 21:56:18.046698 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.046655    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d"} err="failed to get container status \"04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d\": rpc error: code = NotFound desc = could not find container \"04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d\": container with ID starting with 04556f4afdc9322a8351237309247b54a26339a6e50a0a261518d013236ab52d not found: ID does not exist"
Oct 30 21:56:18.095146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.095113    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"]
Oct 30 21:56:18.107057 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.107024    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-58fea9/premium-w0-58fea9-kserve-8698dc969b-8hf4l"]
Oct 30 21:56:18.486118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.486072    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:18.770558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.770451    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:18.825202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.825161    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:18.825406 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:18.825167    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:19.344095 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:19.344060    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" path="/var/lib/kubelet/pods/b9ee38ff-372c-424c-892b-a3b4880e0c34/volumes"
Oct 30 21:56:19.964859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:19.964805    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:56:19.997783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:19.997760    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:56:20.028404 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.028375    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume\") pod \"b526a0b0-b173-4352-8c9e-e868c1416071\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") "
Oct 30 21:56:20.028588 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.028445    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68\" (UniqueName: \"kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68\") pod \"b526a0b0-b173-4352-8c9e-e868c1416071\" (UID: \"b526a0b0-b173-4352-8c9e-e868c1416071\") "
Oct 30 21:56:20.028857 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.028815    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume" pod "b526a0b0-b173-4352-8c9e-e868c1416071" (UID: "b526a0b0-b173-4352-8c9e-e868c1416071"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:20.028949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.028920    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b526a0b0-b173-4352-8c9e-e868c1416071-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:20.030883 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.030854    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68" pod "b526a0b0-b173-4352-8c9e-e868c1416071" (UID: "b526a0b0-b173-4352-8c9e-e868c1416071"). InnerVolumeSpecName "kube-api-access-mff68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:20.086296 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.086257    4549 generic.go:356] "Generic (PLEG): container finished" podID="b526a0b0-b173-4352-8c9e-e868c1416071" containerID="e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173" exitCode=0
Oct 30 21:56:20.086644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.086620    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" event={"ID":"b526a0b0-b173-4352-8c9e-e868c1416071","Type":"ContainerDied","Data":"e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173"}
Oct 30 21:56:20.086879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.086859    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm" event={"ID":"b526a0b0-b173-4352-8c9e-e868c1416071","Type":"ContainerDied","Data":"7021223440d9782c0802f730cd5ac3e360502401fefbd0bf3de4e22d24de910e"}
Oct 30 21:56:20.087014 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.087001    4549 scope.go:122] "RemoveContainer" containerID="e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173"
Oct 30 21:56:20.087334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.087315    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"
Oct 30 21:56:20.088318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.088280    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:20.090246 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.090216    4549 generic.go:356] "Generic (PLEG): container finished" podID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerID="02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227" exitCode=0
Oct 30 21:56:20.090400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.090383    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" event={"ID":"ffda1237-0394-47d7-bcbb-fddae855ef6d","Type":"ContainerDied","Data":"02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227"}
Oct 30 21:56:20.090479 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.090439    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94" event={"ID":"ffda1237-0394-47d7-bcbb-fddae855ef6d","Type":"ContainerDied","Data":"48d14d12e8000846483e8ce7b53d9ccd75b2bee845beb0f7eed50ef5e775f708"}
Oct 30 21:56:20.090557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.090542    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"
Oct 30 21:56:20.099483 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.099461    4549 scope.go:122] "RemoveContainer" containerID="e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173"
Oct 30 21:56:20.099900 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:20.099876    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173\": container with ID starting with e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173 not found: ID does not exist" containerID="e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173"
Oct 30 21:56:20.099988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.099906    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173"} err="failed to get container status \"e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173\": rpc error: code = NotFound desc = could not find container \"e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173\": container with ID starting with e8a90f25e3e2e24c318d32461e8d60a2f31a84100257328903bbc62195f04173 not found: ID does not exist"
Oct 30 21:56:20.099988 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.099923    4549 scope.go:122] "RemoveContainer" containerID="02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227"
Oct 30 21:56:20.110691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.110666    4549 scope.go:122] "RemoveContainer" containerID="02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227"
Oct 30 21:56:20.111146 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:20.111123    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227\": container with ID starting with 02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227 not found: ID does not exist" containerID="02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227"
Oct 30 21:56:20.111264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.111154    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227"} err="failed to get container status \"02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227\": rpc error: code = NotFound desc = could not find container \"02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227\": container with ID starting with 02fbd8360804678ea19a1c7138980f621e17f4d847f0b9c1dc1ac8d493dfe227 not found: ID does not exist"
Oct 30 21:56:20.130167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.130139    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume\") pod \"ffda1237-0394-47d7-bcbb-fddae855ef6d\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") "
Oct 30 21:56:20.130293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.130247    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh\" (UniqueName: \"kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh\") pod \"ffda1237-0394-47d7-bcbb-fddae855ef6d\" (UID: \"ffda1237-0394-47d7-bcbb-fddae855ef6d\") "
Oct 30 21:56:20.130567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.130541    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume" pod "ffda1237-0394-47d7-bcbb-fddae855ef6d" (UID: "ffda1237-0394-47d7-bcbb-fddae855ef6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:20.130721 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.130702    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mff68\" (UniqueName: \"kubernetes.io/projected/b526a0b0-b173-4352-8c9e-e868c1416071-kube-api-access-mff68\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:20.130796 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.130725    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffda1237-0394-47d7-bcbb-fddae855ef6d-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:20.132504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.132483    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh" pod "ffda1237-0394-47d7-bcbb-fddae855ef6d" (UID: "ffda1237-0394-47d7-bcbb-fddae855ef6d"). InnerVolumeSpecName "kube-api-access-q4bvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:20.167331 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.167300    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"]
Oct 30 21:56:20.174322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.174293    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm"]
Oct 30 21:56:20.231904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.231801    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4bvh\" (UniqueName: \"kubernetes.io/projected/ffda1237-0394-47d7-bcbb-fddae855ef6d-kube-api-access-q4bvh\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:20.481689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.481659    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"]
Oct 30 21:56:20.497217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.497191    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94"]
Oct 30 21:56:20.994293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:20.994239    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs" podUID="6a448953-34f7-4a32-985c-242827d5d848" containerName="payload-pre-processing" containerID="cri-o://50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb" gracePeriod=25
Oct 30 21:56:21.038544 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.038509    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:21.284701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.284674    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:56:21.292157 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.292119    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"]
Oct 30 21:56:21.292352 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.292329    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy" containerID="cri-o://6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93" gracePeriod=30
Oct 30 21:56:21.327650 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.327617    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:21.345003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.344974    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume\") pod \"6a448953-34f7-4a32-985c-242827d5d848\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") "
Oct 30 21:56:21.345225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.344971    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" path="/var/lib/kubelet/pods/b526a0b0-b173-4352-8c9e-e868c1416071/volumes"
Oct 30 21:56:21.345225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.345092    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd\" (UniqueName: \"kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd\") pod \"6a448953-34f7-4a32-985c-242827d5d848\" (UID: \"6a448953-34f7-4a32-985c-242827d5d848\") "
Oct 30 21:56:21.345488 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.345458    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume" pod "6a448953-34f7-4a32-985c-242827d5d848" (UID: "6a448953-34f7-4a32-985c-242827d5d848"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:21.345709 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.345688    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a448953-34f7-4a32-985c-242827d5d848-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:21.345775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.345755    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" path="/var/lib/kubelet/pods/ffda1237-0394-47d7-bcbb-fddae855ef6d/volumes"
Oct 30 21:56:21.347951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.347921    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd" pod "6a448953-34f7-4a32-985c-242827d5d848" (UID: "6a448953-34f7-4a32-985c-242827d5d848"). InnerVolumeSpecName "kube-api-access-vn5nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:21.446911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.446874    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn5nd\" (UniqueName: \"kubernetes.io/projected/6a448953-34f7-4a32-985c-242827d5d848-kube-api-access-vn5nd\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:21.618403 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:21.618370    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:22.112120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.112069    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:22.120035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.119999    4549 generic.go:356] "Generic (PLEG): container finished" podID="6a448953-34f7-4a32-985c-242827d5d848" containerID="50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb" exitCode=0
Oct 30 21:56:22.120242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.120042    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs" event={"ID":"6a448953-34f7-4a32-985c-242827d5d848","Type":"ContainerDied","Data":"50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb"}
Oct 30 21:56:22.120242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.120048    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"
Oct 30 21:56:22.120242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.120090    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs" event={"ID":"6a448953-34f7-4a32-985c-242827d5d848","Type":"ContainerDied","Data":"e3bd7e7d2ddbeb2fde3be5616ba2c528dc9db5c1cc3fc4d495440774ef2c1ccf"}
Oct 30 21:56:22.120242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.120114    4549 scope.go:122] "RemoveContainer" containerID="50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb"
Oct 30 21:56:22.132000 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.131980    4549 scope.go:122] "RemoveContainer" containerID="50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb"
Oct 30 21:56:22.132381 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:22.132357    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb\": container with ID starting with 50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb not found: ID does not exist" containerID="50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb"
Oct 30 21:56:22.132470 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.132400    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb"} err="failed to get container status \"50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb\": rpc error: code = NotFound desc = could not find container \"50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb\": container with ID starting with 50ea42380cdb99e9b29b13efd0bab4bf8ccd004533983f4e6eb420c5dfa91adb not found: ID does not exist"
Oct 30 21:56:22.173491 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.173454    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"]
Oct 30 21:56:22.185335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:22.185299    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs"]
Oct 30 21:56:23.345356 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:23.345323    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a448953-34f7-4a32-985c-242827d5d848" path="/var/lib/kubelet/pods/6a448953-34f7-4a32-985c-242827d5d848/volumes"
Oct 30 21:56:24.512769 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:24.512731    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:24.735298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:24.735256    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:24.776937 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:24.776814    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:25.009744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.009708    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:25.241291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.241247    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" containerID="cri-o://e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62" gracePeriod=45
Oct 30 21:56:25.527325 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.527297    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:56:25.614083 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614044    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs\" (UniqueName: \"kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs\") pod \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") "
Oct 30 21:56:25.614277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614134    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home\") pod \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") "
Oct 30 21:56:25.614277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614177    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m\" (UniqueName: \"kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m\") pod \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") "
Oct 30 21:56:25.614277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614245    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm\") pod \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") "
Oct 30 21:56:25.614471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614314    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache\") pod \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") "
Oct 30 21:56:25.614471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614369    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir\") pod \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\" (UID: \"b562bdeb-bf87-4c6f-b22e-34792cce8cc8\") "
Oct 30 21:56:25.614471 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614439    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home" pod "b562bdeb-bf87-4c6f-b22e-34792cce8cc8" (UID: "b562bdeb-bf87-4c6f-b22e-34792cce8cc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:25.614657 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614626    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache" pod "b562bdeb-bf87-4c6f-b22e-34792cce8cc8" (UID: "b562bdeb-bf87-4c6f-b22e-34792cce8cc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:25.614775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614753    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir" pod "b562bdeb-bf87-4c6f-b22e-34792cce8cc8" (UID: "b562bdeb-bf87-4c6f-b22e-34792cce8cc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:25.614915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614889    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:25.614915 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614911    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:25.615046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.614924    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:25.617196 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.617165    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m" pod "b562bdeb-bf87-4c6f-b22e-34792cce8cc8" (UID: "b562bdeb-bf87-4c6f-b22e-34792cce8cc8"). InnerVolumeSpecName "kube-api-access-z496m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:25.617379 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.617350    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm" pod "b562bdeb-bf87-4c6f-b22e-34792cce8cc8" (UID: "b562bdeb-bf87-4c6f-b22e-34792cce8cc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:25.617538 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.617409    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs" pod "b562bdeb-bf87-4c6f-b22e-34792cce8cc8" (UID: "b562bdeb-bf87-4c6f-b22e-34792cce8cc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:25.715720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.715685    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:25.715720 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.715718    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z496m\" (UniqueName: \"kubernetes.io/projected/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-kube-api-access-z496m\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:25.715949 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:25.715732    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b562bdeb-bf87-4c6f-b22e-34792cce8cc8-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.020585 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.020550    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:56:26.021011 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.020988    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:56:26.147249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.147164    4549 generic.go:356] "Generic (PLEG): container finished" podID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerID="e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62" exitCode=0
Oct 30 21:56:26.147249 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.147214    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"
Oct 30 21:56:26.147506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.147235    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" event={"ID":"b562bdeb-bf87-4c6f-b22e-34792cce8cc8","Type":"ContainerDied","Data":"e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62"}
Oct 30 21:56:26.147506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.147295    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" event={"ID":"b562bdeb-bf87-4c6f-b22e-34792cce8cc8","Type":"ContainerDied","Data":"53b3d32958d1c32ce7cb548855d486d627b1ac0e3f57805d383bb70b04cef758"}
Oct 30 21:56:26.147506 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.147321    4549 scope.go:122] "RemoveContainer" containerID="e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62"
Oct 30 21:56:26.158581 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.158523    4549 scope.go:122] "RemoveContainer" containerID="e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62"
Oct 30 21:56:26.159197 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:26.159123    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62\": container with ID starting with e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62 not found: ID does not exist" containerID="e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62"
Oct 30 21:56:26.159197 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.159167    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62"} err="failed to get container status \"e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62\": rpc error: code = NotFound desc = could not find container \"e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62\": container with ID starting with e82637214229f543656dce9eca66632b2ac5c06acac38f8d36dda35e2dfbbf62 not found: ID does not exist"
Oct 30 21:56:26.203866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.203818    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"]
Oct 30 21:56:26.210816 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.210788    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6"]
Oct 30 21:56:26.519070 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.518996    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-56ac74/distinct-w1-56ac74-kserve-698db5b784-4kwh6" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:56:26.576707 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:26.576673    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:56:26.577776 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.577748    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:56:26.707964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.707866    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:26.729099 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729066    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729177    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729206    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729253    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729279    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729352    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729425    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729451    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729479    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:26.729534 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729494    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket\") pod \"bb817463-80b7-427d-b20f-5a1c6914f87d\" (UID: \"bb817463-80b7-427d-b20f-5a1c6914f87d\") "
Oct 30 21:56:26.729823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729553    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:26.729823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729734    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:26.729823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729808    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:26.730309 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.729987    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:26.730428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.730338    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb817463-80b7-427d-b20f-5a1c6914f87d-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.730428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.730361    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.730428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.730375    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.730428 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.730388    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.731774 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.731750    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:26.731898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.731876    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "kube-api-access-z7lsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:26.731953 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.731925    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:56:26.732872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.732818    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token" pod "bb817463-80b7-427d-b20f-5a1c6914f87d" (UID: "bb817463-80b7-427d-b20f-5a1c6914f87d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:26.831929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.831890    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.831929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.831920    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.831929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.831930    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.831929 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.831940    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7lsn\" (UniqueName: \"kubernetes.io/projected/bb817463-80b7-427d-b20f-5a1c6914f87d-kube-api-access-z7lsn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:26.832228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:26.831950    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb817463-80b7-427d-b20f-5a1c6914f87d-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:27.155060 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.155025    4549 generic.go:356] "Generic (PLEG): container finished" podID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerID="6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93" exitCode=0
Oct 30 21:56:27.155261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.155090    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" event={"ID":"bb817463-80b7-427d-b20f-5a1c6914f87d","Type":"ContainerDied","Data":"6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93"}
Oct 30 21:56:27.155261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.155111    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"
Oct 30 21:56:27.155261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.155154    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq" event={"ID":"bb817463-80b7-427d-b20f-5a1c6914f87d","Type":"ContainerDied","Data":"a9bf6ddd131f5a51a347ede326f23b86f17a441e5688fa019400f53ad666c79f"}
Oct 30 21:56:27.155261 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.155181    4549 scope.go:122] "RemoveContainer" containerID="6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93"
Oct 30 21:56:27.165725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.165704    4549 scope.go:122] "RemoveContainer" containerID="01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"
Oct 30 21:56:27.177527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.177502    4549 scope.go:122] "RemoveContainer" containerID="6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93"
Oct 30 21:56:27.177872 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:27.177851    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93\": container with ID starting with 6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93 not found: ID does not exist" containerID="6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93"
Oct 30 21:56:27.177974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.177889    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93"} err="failed to get container status \"6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93\": rpc error: code = NotFound desc = could not find container \"6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93\": container with ID starting with 6bfee7602875d4ff7a24f11530bddb21bf7b7f03c0699ab72da5a4596c70dd93 not found: ID does not exist"
Oct 30 21:56:27.177974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.177914    4549 scope.go:122] "RemoveContainer" containerID="01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"
Oct 30 21:56:27.178221 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:27.178198    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157\": container with ID starting with 01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157 not found: ID does not exist" containerID="01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"
Oct 30 21:56:27.178268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.178231    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157"} err="failed to get container status \"01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157\": rpc error: code = NotFound desc = could not find container \"01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157\": container with ID starting with 01860ac3a4dee878f42be5d8886428cfc4af4ab052ca1fe8a6d7f1d7d924e157 not found: ID does not exist"
Oct 30 21:56:27.237071 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.237030    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"]
Oct 30 21:56:27.239557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.239530    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq"]
Oct 30 21:56:27.344482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.344446    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" path="/var/lib/kubelet/pods/b562bdeb-bf87-4c6f-b22e-34792cce8cc8/volumes"
Oct 30 21:56:27.344968 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:27.344952    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" path="/var/lib/kubelet/pods/bb817463-80b7-427d-b20f-5a1c6914f87d/volumes"
Oct 30 21:56:28.692229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:28.692146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:30.208554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:30.208520    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:31.913504 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:31.913466    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:32.425082 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:32.425049    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"]
Oct 30 21:56:32.467900 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:32.467855    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"]
Oct 30 21:56:32.599268 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:32.599242    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 21:56:32.651536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:32.651505    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:32.829607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:32.829581    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 21:56:32.894600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:32.894569    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.036749 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.036714    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z_d1482a83-e8a0-40a9-86de-755c84032fc4/payload-processing/0.log"
Oct 30 21:56:33.092374 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.092302    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:56:33.213187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.213151    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume\") pod \"b36205f5-889f-4f2d-b0c7-8940b127181f\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") "
Oct 30 21:56:33.213402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.213287    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm\" (UniqueName: \"kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm\") pod \"b36205f5-889f-4f2d-b0c7-8940b127181f\" (UID: \"b36205f5-889f-4f2d-b0c7-8940b127181f\") "
Oct 30 21:56:33.213537 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.213515    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume" pod "b36205f5-889f-4f2d-b0c7-8940b127181f" (UID: "b36205f5-889f-4f2d-b0c7-8940b127181f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:33.213679 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.213661    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36205f5-889f-4f2d-b0c7-8940b127181f-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:33.217085 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.217054    4549 generic.go:356] "Generic (PLEG): container finished" podID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerID="3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25" exitCode=0
Oct 30 21:56:33.217220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.217102    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"
Oct 30 21:56:33.217220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.217132    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" event={"ID":"b36205f5-889f-4f2d-b0c7-8940b127181f","Type":"ContainerDied","Data":"3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25"}
Oct 30 21:56:33.217220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.217183    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db" event={"ID":"b36205f5-889f-4f2d-b0c7-8940b127181f","Type":"ContainerDied","Data":"558c7668bf56a5e7320f16c31e4594f4bebcd1792788d6ac80cb93faedfaee8d"}
Oct 30 21:56:33.217220 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.217204    4549 scope.go:122] "RemoveContainer" containerID="3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25"
Oct 30 21:56:33.222160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.222128    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm" pod "b36205f5-889f-4f2d-b0c7-8940b127181f" (UID: "b36205f5-889f-4f2d-b0c7-8940b127181f"). InnerVolumeSpecName "kube-api-access-p2jxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:33.229683 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.229664    4549 scope.go:122] "RemoveContainer" containerID="3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25"
Oct 30 21:56:33.230014 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:33.229993    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25\": container with ID starting with 3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25 not found: ID does not exist" containerID="3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25"
Oct 30 21:56:33.230074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.230024    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25"} err="failed to get container status \"3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25\": rpc error: code = NotFound desc = could not find container \"3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25\": container with ID starting with 3a6cee83223e791f4dbdff2604b5044a8c9e91b87b3ec1abe8ef87ab600c9e25 not found: ID does not exist"
Oct 30 21:56:33.314344 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.314311    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2jxm\" (UniqueName: \"kubernetes.io/projected/b36205f5-889f-4f2d-b0c7-8940b127181f-kube-api-access-p2jxm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:33.558818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.558785    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"]
Oct 30 21:56:33.569705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.569672    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db"]
Oct 30 21:56:33.830372 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:33.830344    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:35.343821 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:35.343784    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" path="/var/lib/kubelet/pods/b36205f5-889f-4f2d-b0c7-8940b127181f/volumes"
Oct 30 21:56:35.889897 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:35.889860    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"]
Oct 30 21:56:36.041096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:36.041060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:36.210204 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:36.210118    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:37.426405 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:37.426339    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerName="payload-processing" containerID="cri-o://eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7" gracePeriod=25
Oct 30 21:56:37.468930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:37.468883    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerName="payload-pre-processing" containerID="cri-o://072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632" gracePeriod=25
Oct 30 21:56:37.737770 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:37.737676    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:38.757333 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:38.757302    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:38.935443 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:38.935340    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:39.170146 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.170102    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"]
Oct 30 21:56:39.170387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.170358    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp" podUID="eb7ca725-d313-455a-89dc-9da4580a8755" containerName="maas-api" containerID="cri-o://f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f" gracePeriod=30
Oct 30 21:56:39.272230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.272197    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:39.431283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.431218    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:56:39.494444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.494406    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls\") pod \"eb7ca725-d313-455a-89dc-9da4580a8755\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") "
Oct 30 21:56:39.494871 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.494799    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs\") pod \"eb7ca725-d313-455a-89dc-9da4580a8755\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") "
Oct 30 21:56:39.494996 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.494868    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f\" (UniqueName: \"kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f\") pod \"eb7ca725-d313-455a-89dc-9da4580a8755\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") "
Oct 30 21:56:39.495086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.495056    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca\") pod \"eb7ca725-d313-455a-89dc-9da4580a8755\" (UID: \"eb7ca725-d313-455a-89dc-9da4580a8755\") "
Oct 30 21:56:39.495600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.495562    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca" pod "eb7ca725-d313-455a-89dc-9da4580a8755" (UID: "eb7ca725-d313-455a-89dc-9da4580a8755"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:39.495982 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.495961    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7ca725-d313-455a-89dc-9da4580a8755-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:39.497528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.497496    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls" pod "eb7ca725-d313-455a-89dc-9da4580a8755" (UID: "eb7ca725-d313-455a-89dc-9da4580a8755"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:39.498170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.498145    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs" pod "eb7ca725-d313-455a-89dc-9da4580a8755" (UID: "eb7ca725-d313-455a-89dc-9da4580a8755"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:39.498295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.498270    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f" pod "eb7ca725-d313-455a-89dc-9da4580a8755" (UID: "eb7ca725-d313-455a-89dc-9da4580a8755"). InnerVolumeSpecName "kube-api-access-nhz7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:39.597319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.597272    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:39.597319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.597305    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhz7f\" (UniqueName: \"kubernetes.io/projected/eb7ca725-d313-455a-89dc-9da4580a8755-kube-api-access-nhz7f\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:39.597319 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.597318    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb7ca725-d313-455a-89dc-9da4580a8755-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:39.871964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:39.871922    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:40.048468 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.048414    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:40.286490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.286391    4549 generic.go:356] "Generic (PLEG): container finished" podID="eb7ca725-d313-455a-89dc-9da4580a8755" containerID="f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f" exitCode=0
Oct 30 21:56:40.286490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.286441    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"
Oct 30 21:56:40.286490 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.286457    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp" event={"ID":"eb7ca725-d313-455a-89dc-9da4580a8755","Type":"ContainerDied","Data":"f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f"}
Oct 30 21:56:40.286767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.286519    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp" event={"ID":"eb7ca725-d313-455a-89dc-9da4580a8755","Type":"ContainerDied","Data":"31d682746d3a4e884c54e6b3b20b5bc21d4bad67a83d4774f9d4b8de894ebeb5"}
Oct 30 21:56:40.286767 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.286544    4549 scope.go:122] "RemoveContainer" containerID="f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f"
Oct 30 21:56:40.298913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.298884    4549 scope.go:122] "RemoveContainer" containerID="f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f"
Oct 30 21:56:40.299243 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:40.299219    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f\": container with ID starting with f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f not found: ID does not exist" containerID="f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f"
Oct 30 21:56:40.299329 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.299261    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f"} err="failed to get container status \"f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f\": rpc error: code = NotFound desc = could not find container \"f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f\": container with ID starting with f65f95cc144302f17eec0567e9515df809f56948a66574228085ae772692a11f not found: ID does not exist"
Oct 30 21:56:40.359017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.358985    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"]
Oct 30 21:56:40.383889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.383855    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-56ac74-657c65cd76-l5zbp"]
Oct 30 21:56:40.558106 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.558008    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:40.890946 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:40.890904    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerName="payload-pre-processing" containerID="cri-o://c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601" gracePeriod=25
Oct 30 21:56:41.343869 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:41.343813    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb7ca725-d313-455a-89dc-9da4580a8755" path="/var/lib/kubelet/pods/eb7ca725-d313-455a-89dc-9da4580a8755/volumes"
Oct 30 21:56:41.867632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:41.867542    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:41.958844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:41.958791    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:41.958844 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:41.958823    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:42.296598 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:42.296485    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:43.684631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:43.684589    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:45.498911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.498823    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"]
Oct 30 21:56:45.499449 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.499082    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" podUID="abe73d82-7ffe-44bb-a9fe-ece88010a356" containerName="maas-api" containerID="cri-o://bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98" gracePeriod=30
Oct 30 21:56:45.588335 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.588302    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:45.799964 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.799935    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:56:45.873903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.872693    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\") pod \"abe73d82-7ffe-44bb-a9fe-ece88010a356\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") "
Oct 30 21:56:45.873903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.872783    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c\" (UniqueName: \"kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c\") pod \"abe73d82-7ffe-44bb-a9fe-ece88010a356\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") "
Oct 30 21:56:45.873903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.872862    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs\") pod \"abe73d82-7ffe-44bb-a9fe-ece88010a356\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") "
Oct 30 21:56:45.873903 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.872910    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca\") pod \"abe73d82-7ffe-44bb-a9fe-ece88010a356\" (UID: \"abe73d82-7ffe-44bb-a9fe-ece88010a356\") "
Oct 30 21:56:45.874365 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.873947    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca" pod "abe73d82-7ffe-44bb-a9fe-ece88010a356" (UID: "abe73d82-7ffe-44bb-a9fe-ece88010a356"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:45.889861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.880362    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c" pod "abe73d82-7ffe-44bb-a9fe-ece88010a356" (UID: "abe73d82-7ffe-44bb-a9fe-ece88010a356"). InnerVolumeSpecName "kube-api-access-q6l9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:45.896121 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.896056    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls" pod "abe73d82-7ffe-44bb-a9fe-ece88010a356" (UID: "abe73d82-7ffe-44bb-a9fe-ece88010a356"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:45.903855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.899442    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs" pod "abe73d82-7ffe-44bb-a9fe-ece88010a356" (UID: "abe73d82-7ffe-44bb-a9fe-ece88010a356"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:45.974349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.974313    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6l9c\" (UniqueName: \"kubernetes.io/projected/abe73d82-7ffe-44bb-a9fe-ece88010a356-kube-api-access-q6l9c\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:45.974349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.974347    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:45.974349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.974356    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe73d82-7ffe-44bb-a9fe-ece88010a356-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:45.974602 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:45.974365    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abe73d82-7ffe-44bb-a9fe-ece88010a356-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:46.105338 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.105290    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:46.297526 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.297495    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:46.326654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.326619    4549 generic.go:356] "Generic (PLEG): container finished" podID="abe73d82-7ffe-44bb-a9fe-ece88010a356" containerID="bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98" exitCode=0
Oct 30 21:56:46.326914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.326670    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"
Oct 30 21:56:46.326914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.326709    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" event={"ID":"abe73d82-7ffe-44bb-a9fe-ece88010a356","Type":"ContainerDied","Data":"bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98"}
Oct 30 21:56:46.326914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.326777    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp" event={"ID":"abe73d82-7ffe-44bb-a9fe-ece88010a356","Type":"ContainerDied","Data":"8ee23417e18dd9d150da863ee21a60ee56822aa4fe325768d198602a1c58ed22"}
Oct 30 21:56:46.326914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.326800    4549 scope.go:122] "RemoveContainer" containerID="bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98"
Oct 30 21:56:46.337673 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.337649    4549 scope.go:122] "RemoveContainer" containerID="bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98"
Oct 30 21:56:46.341675 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:46.341644    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98\": container with ID starting with bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98 not found: ID does not exist" containerID="bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98"
Oct 30 21:56:46.341805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.341684    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98"} err="failed to get container status \"bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98\": rpc error: code = NotFound desc = could not find container \"bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98\": container with ID starting with bdf69723307e85c9478a880c36c565e4e1d2e1fa0d683cdb95f0ab43f2f74a98 not found: ID does not exist"
Oct 30 21:56:46.388463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.388374    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"]
Oct 30 21:56:46.391554 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.391529    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-58fea9-6856c8bb8c-bwqdp"]
Oct 30 21:56:46.945777 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:46.945728    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:47.342674 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:47.342643    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abe73d82-7ffe-44bb-a9fe-ece88010a356" path="/var/lib/kubelet/pods/abe73d82-7ffe-44bb-a9fe-ece88010a356/volumes"
Oct 30 21:56:48.135675 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:48.135634    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:48.203881 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:48.203815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:48.822187 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:48.822150    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:49.121530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:49.121424    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:49.567494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:49.567459    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:49.881168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:49.881077    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:50.825562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:50.825533    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:51.058101 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:51.058066    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:51.293794 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:51.293707    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:51.597997 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:51.597962    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:51.959557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:51.959472    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:51.959557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:51.959511    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:53.058208 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.058176    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"]
Oct 30 21:56:53.058751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.058439    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" podUID="59724577-6c37-4b72-a404-725cb32ff7b3" containerName="maas-api" containerID="cri-o://bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5" gracePeriod=30
Oct 30 21:56:53.219714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.219681    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:53.323025 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.322999    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:56:53.369853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.367681    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs\") pod \"59724577-6c37-4b72-a404-725cb32ff7b3\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") "
Oct 30 21:56:53.369853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.367733    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\") pod \"59724577-6c37-4b72-a404-725cb32ff7b3\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") "
Oct 30 21:56:53.369853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.367872    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca\") pod \"59724577-6c37-4b72-a404-725cb32ff7b3\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") "
Oct 30 21:56:53.418086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.418049    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv\" (UniqueName: \"kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv\") pod \"59724577-6c37-4b72-a404-725cb32ff7b3\" (UID: \"59724577-6c37-4b72-a404-725cb32ff7b3\") "
Oct 30 21:56:53.418281 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.418210    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca" pod "59724577-6c37-4b72-a404-725cb32ff7b3" (UID: "59724577-6c37-4b72-a404-725cb32ff7b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:53.418649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.418624    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59724577-6c37-4b72-a404-725cb32ff7b3-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:53.421548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.421520    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv" pod "59724577-6c37-4b72-a404-725cb32ff7b3" (UID: "59724577-6c37-4b72-a404-725cb32ff7b3"). InnerVolumeSpecName "kube-api-access-dcdnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:53.421861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.421802    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls" pod "59724577-6c37-4b72-a404-725cb32ff7b3" (UID: "59724577-6c37-4b72-a404-725cb32ff7b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:53.423035 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.423008    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs" pod "59724577-6c37-4b72-a404-725cb32ff7b3" (UID: "59724577-6c37-4b72-a404-725cb32ff7b3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:53.438209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.438181    4549 generic.go:356] "Generic (PLEG): container finished" podID="59724577-6c37-4b72-a404-725cb32ff7b3" containerID="bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5" exitCode=0
Oct 30 21:56:53.438425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.438261    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" event={"ID":"59724577-6c37-4b72-a404-725cb32ff7b3","Type":"ContainerDied","Data":"bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5"}
Oct 30 21:56:53.438561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.438453    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8" event={"ID":"59724577-6c37-4b72-a404-725cb32ff7b3","Type":"ContainerDied","Data":"becc99bf33ca6b56bdf59551acdd1a179f409bf999de8fe5fb632589da7352ab"}
Oct 30 21:56:53.438561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.438478    4549 scope.go:122] "RemoveContainer" containerID="bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5"
Oct 30 21:56:53.438561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.438338    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"
Oct 30 21:56:53.449677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.449637    4549 scope.go:122] "RemoveContainer" containerID="bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5"
Oct 30 21:56:53.450088 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:53.450062    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5\": container with ID starting with bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5 not found: ID does not exist" containerID="bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5"
Oct 30 21:56:53.450191 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.450100    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5"} err="failed to get container status \"bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5\": rpc error: code = NotFound desc = could not find container \"bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5\": container with ID starting with bfb4f74c7958c364786be7eb990b5b664d2f071d70f9488f72ce7814dc8a6dd5 not found: ID does not exist"
Oct 30 21:56:53.502420 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.502355    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"]
Oct 30 21:56:53.542877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.540115    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:53.542877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.540153    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59724577-6c37-4b72-a404-725cb32ff7b3-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:53.542877 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.540168    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcdnv\" (UniqueName: \"kubernetes.io/projected/59724577-6c37-4b72-a404-725cb32ff7b3-kube-api-access-dcdnv\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:53.545890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.544443    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-55c5e0-6b9579cdc4-kshk8"]
Oct 30 21:56:53.842303 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:53.842263    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:54.083535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:54.083500    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:54.399632 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:54.399594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:54.655277 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:54.655194    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:55.342706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:55.342658    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59724577-6c37-4b72-a404-725cb32ff7b3" path="/var/lib/kubelet/pods/59724577-6c37-4b72-a404-725cb32ff7b3/volumes"
Oct 30 21:56:56.591352 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:56.591308    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:56:56.667959 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:56.667928    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:56.863453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:56.863356    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:57.222397 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.222301    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:57.777805 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.777768    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:56:57.793059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.793023    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn\" (UniqueName: \"kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn\") pod \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") "
Oct 30 21:56:57.793273 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.793169    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume\") pod \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\" (UID: \"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d\") "
Oct 30 21:56:57.793603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.793581    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume" pod "9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" (UID: "9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:57.795419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.795388    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn" pod "9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" (UID: "9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d"). InnerVolumeSpecName "kube-api-access-pk4kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:57.813260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.813233    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:56:57.894188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.894148    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume\") pod \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") "
Oct 30 21:56:57.894401 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.894237    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq\" (UniqueName: \"kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq\") pod \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\" (UID: \"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e\") "
Oct 30 21:56:57.894603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.894569    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume" pod "41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" (UID: "41cdfe49-fc8c-43a9-a7a5-3515e5626c6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:57.894751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.894683    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk4kn\" (UniqueName: \"kubernetes.io/projected/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-kube-api-access-pk4kn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:57.894751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.894708    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:57.894751 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.894722    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:57.896688 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.896661    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq" pod "41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" (UID: "41cdfe49-fc8c-43a9-a7a5-3515e5626c6e"). InnerVolumeSpecName "kube-api-access-b6ghq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:57.995699 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:57.995657    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6ghq\" (UniqueName: \"kubernetes.io/projected/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e-kube-api-access-b6ghq\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:56:58.135501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.135461    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:58.468568 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.468526    4549 generic.go:356] "Generic (PLEG): container finished" podID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerID="072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632" exitCode=0
Oct 30 21:56:58.468786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.468572    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"
Oct 30 21:56:58.468786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.468601    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" event={"ID":"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e","Type":"ContainerDied","Data":"072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632"}
Oct 30 21:56:58.468786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.468651    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq" event={"ID":"41cdfe49-fc8c-43a9-a7a5-3515e5626c6e","Type":"ContainerDied","Data":"78e11666ce0c7798b846a85e41b0e881b2f4667b1352e9636a58c652e30d3683"}
Oct 30 21:56:58.468786 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.468664    4549 scope.go:122] "RemoveContainer" containerID="072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632"
Oct 30 21:56:58.470363 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.470334    4549 generic.go:356] "Generic (PLEG): container finished" podID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerID="eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7" exitCode=0
Oct 30 21:56:58.470513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.470495    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" event={"ID":"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d","Type":"ContainerDied","Data":"eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7"}
Oct 30 21:56:58.470594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.470543    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5" event={"ID":"9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d","Type":"ContainerDied","Data":"5a7175c368a178c3822831f6fc7af737e315043e8166f1129bbc50c53bdf260e"}
Oct 30 21:56:58.470594 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.470561    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"
Oct 30 21:56:58.480346 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.480323    4549 scope.go:122] "RemoveContainer" containerID="072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632"
Oct 30 21:56:58.480728 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:58.480700    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632\": container with ID starting with 072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632 not found: ID does not exist" containerID="072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632"
Oct 30 21:56:58.480864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.480742    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632"} err="failed to get container status \"072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632\": rpc error: code = NotFound desc = could not find container \"072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632\": container with ID starting with 072f5c8d232f9d8eb02f714b56b8b0b8e70d6102763fa4002f7e6d76ab1a8632 not found: ID does not exist"
Oct 30 21:56:58.480864 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.480776    4549 scope.go:122] "RemoveContainer" containerID="eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7"
Oct 30 21:56:58.490607 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.490585    4549 scope.go:122] "RemoveContainer" containerID="eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7"
Oct 30 21:56:58.490967 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:56:58.490946    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7\": container with ID starting with eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7 not found: ID does not exist" containerID="eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7"
Oct 30 21:56:58.491055 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.490977    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7"} err="failed to get container status \"eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7\": rpc error: code = NotFound desc = could not find container \"eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7\": container with ID starting with eaa63eabe824c270e46c0f083da3e182afe8a1ac2574c5efd38e83d9de29c2f7 not found: ID does not exist"
Oct 30 21:56:58.533593 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.533556    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"]
Oct 30 21:56:58.535649 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.535621    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5"]
Oct 30 21:56:58.579358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.579317    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"]
Oct 30 21:56:58.593139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.593104    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq"]
Oct 30 21:56:58.909992 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:58.909958    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:59.342645 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:59.342610    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" path="/var/lib/kubelet/pods/41cdfe49-fc8c-43a9-a7a5-3515e5626c6e/volumes"
Oct 30 21:56:59.343069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:56:59.343054    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" path="/var/lib/kubelet/pods/9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d/volumes"
Oct 30 21:57:00.803722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:00.803682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:01.081742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.081707    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"]
Oct 30 21:57:01.082643 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.082603    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy" containerID="cri-o://f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a" gracePeriod=30
Oct 30 21:57:01.096873 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.096808    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:01.170154 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.170117    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:57:01.230426 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.230393    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm\" (UniqueName: \"kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm\") pod \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") "
Oct 30 21:57:01.230639 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.230596    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume\") pod \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\" (UID: \"42172e4b-91d7-4bab-8aa8-c22212cdfaa7\") "
Oct 30 21:57:01.231289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.231246    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume" pod "42172e4b-91d7-4bab-8aa8-c22212cdfaa7" (UID: "42172e4b-91d7-4bab-8aa8-c22212cdfaa7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:01.233472 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.233441    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm" pod "42172e4b-91d7-4bab-8aa8-c22212cdfaa7" (UID: "42172e4b-91d7-4bab-8aa8-c22212cdfaa7"). InnerVolumeSpecName "kube-api-access-pxmdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:01.332417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.332315    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxmdm\" (UniqueName: \"kubernetes.io/projected/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-kube-api-access-pxmdm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:01.332417 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.332356    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42172e4b-91d7-4bab-8aa8-c22212cdfaa7-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:01.341654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.341618    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:01.490351 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.490318    4549 generic.go:356] "Generic (PLEG): container finished" podID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerID="c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601" exitCode=0
Oct 30 21:57:01.490541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.490371    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"
Oct 30 21:57:01.490541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.490409    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" event={"ID":"42172e4b-91d7-4bab-8aa8-c22212cdfaa7","Type":"ContainerDied","Data":"c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601"}
Oct 30 21:57:01.490541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.490468    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc" event={"ID":"42172e4b-91d7-4bab-8aa8-c22212cdfaa7","Type":"ContainerDied","Data":"9703df7bd1fdac8831e0d7785d71c502ef4a5e3563a684cb723ba07f3fdffd67"}
Oct 30 21:57:01.490541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.490498    4549 scope.go:122] "RemoveContainer" containerID="c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601"
Oct 30 21:57:01.500697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.500674    4549 scope.go:122] "RemoveContainer" containerID="c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601"
Oct 30 21:57:01.501082 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:01.501062    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601\": container with ID starting with c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601 not found: ID does not exist" containerID="c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601"
Oct 30 21:57:01.501150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.501093    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601"} err="failed to get container status \"c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601\": rpc error: code = NotFound desc = could not find container \"c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601\": container with ID starting with c3bd75b03e12a05ee5e350d29385bcafb78df47279ea00564a2a1c83378c4601 not found: ID does not exist"
Oct 30 21:57:01.551147 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.551109    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"]
Oct 30 21:57:01.562886 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.562850    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc"]
Oct 30 21:57:01.573234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:01.573198    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:02.795219 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:02.795174    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:03.184610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:03.184562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:03.342685 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:03.342650    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" path="/var/lib/kubelet/pods/42172e4b-91d7-4bab-8aa8-c22212cdfaa7/volumes"
Oct 30 21:57:04.051668 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.051634    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.235415 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.235376    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.376536 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.376496    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"]
Oct 30 21:57:04.376742 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.376715    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" containerID="cri-o://c5df2e9b9bde63f28ea45fc6974a67f4f542529ff1932b54d36f6d2e68759aa5" gracePeriod=30
Oct 30 21:57:04.425195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.425158    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"]
Oct 30 21:57:04.425425 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.425364    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" podUID="b70988b9-c257-47f2-98de-5eac5f217367" containerName="payload-processing" containerID="cri-o://b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143" gracePeriod=30
Oct 30 21:57:04.429801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.429756    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.457088 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.457047    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"]
Oct 30 21:57:04.457280 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.457258    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" podUID="b469b4b0-6207-42a6-b4e1-a89b457c975f" containerName="payload-pre-processing" containerID="cri-o://5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51" gracePeriod=30
Oct 30 21:57:04.526176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.525928    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" podUID="b469b4b0-6207-42a6-b4e1-a89b457c975f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.219:9004: connect: connection refused"
Oct 30 21:57:04.526176 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.525957    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" podUID="b70988b9-c257-47f2-98de-5eac5f217367" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.220:9004: connect: connection refused"
Oct 30 21:57:04.618234 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.618096    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.718697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.718670    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:57:04.748050 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.748027    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:57:04.775882 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.775795    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7\" (UniqueName: \"kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7\") pod \"b70988b9-c257-47f2-98de-5eac5f217367\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") "
Oct 30 21:57:04.776078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.775950    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume\") pod \"b70988b9-c257-47f2-98de-5eac5f217367\" (UID: \"b70988b9-c257-47f2-98de-5eac5f217367\") "
Oct 30 21:57:04.776431 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.776400    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume" pod "b70988b9-c257-47f2-98de-5eac5f217367" (UID: "b70988b9-c257-47f2-98de-5eac5f217367"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:04.782862 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.782806    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7" pod "b70988b9-c257-47f2-98de-5eac5f217367" (UID: "b70988b9-c257-47f2-98de-5eac5f217367"). InnerVolumeSpecName "kube-api-access-qmwc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:04.836802 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.836666    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.877855 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.877802    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume\") pod \"b469b4b0-6207-42a6-b4e1-a89b457c975f\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") "
Oct 30 21:57:04.878078 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.877919    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn\" (UniqueName: \"kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn\") pod \"b469b4b0-6207-42a6-b4e1-a89b457c975f\" (UID: \"b469b4b0-6207-42a6-b4e1-a89b457c975f\") "
Oct 30 21:57:04.878221 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.878204    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmwc7\" (UniqueName: \"kubernetes.io/projected/b70988b9-c257-47f2-98de-5eac5f217367-kube-api-access-qmwc7\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:04.878284 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.878221    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70988b9-c257-47f2-98de-5eac5f217367-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:04.878349 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.878321    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume" pod "b469b4b0-6207-42a6-b4e1-a89b457c975f" (UID: "b469b4b0-6207-42a6-b4e1-a89b457c975f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:04.880291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.880263    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn" pod "b469b4b0-6207-42a6-b4e1-a89b457c975f" (UID: "b469b4b0-6207-42a6-b4e1-a89b457c975f"). InnerVolumeSpecName "kube-api-access-h6wxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:04.979156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.979068    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b469b4b0-6207-42a6-b4e1-a89b457c975f-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:04.979156 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:04.979099    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6wxn\" (UniqueName: \"kubernetes.io/projected/b469b4b0-6207-42a6-b4e1-a89b457c975f-kube-api-access-h6wxn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:05.048463 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.048426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:05.218505 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.218464    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:05.525047 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.525014    4549 generic.go:356] "Generic (PLEG): container finished" podID="b469b4b0-6207-42a6-b4e1-a89b457c975f" containerID="5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51" exitCode=0
Oct 30 21:57:05.525231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.525062    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"
Oct 30 21:57:05.525231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.525099    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" event={"ID":"b469b4b0-6207-42a6-b4e1-a89b457c975f","Type":"ContainerDied","Data":"5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51"}
Oct 30 21:57:05.525231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.525158    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc" event={"ID":"b469b4b0-6207-42a6-b4e1-a89b457c975f","Type":"ContainerDied","Data":"a9736133cac479b729e069f82dcd585886d1782bec94e78f634510e5cb36f030"}
Oct 30 21:57:05.525231 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.525192    4549 scope.go:122] "RemoveContainer" containerID="5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51"
Oct 30 21:57:05.526610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.526437    4549 generic.go:356] "Generic (PLEG): container finished" podID="b70988b9-c257-47f2-98de-5eac5f217367" containerID="b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143" exitCode=0
Oct 30 21:57:05.526610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.526474    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"
Oct 30 21:57:05.526610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.526494    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" event={"ID":"b70988b9-c257-47f2-98de-5eac5f217367","Type":"ContainerDied","Data":"b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143"}
Oct 30 21:57:05.526610 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.526535    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s" event={"ID":"b70988b9-c257-47f2-98de-5eac5f217367","Type":"ContainerDied","Data":"8dfd60a65c73812db7398922d514269006223f7d37969705b94ac7c807ea3764"}
Oct 30 21:57:05.534748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.534719    4549 scope.go:122] "RemoveContainer" containerID="5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51"
Oct 30 21:57:05.535058 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:05.535015    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51\": container with ID starting with 5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51 not found: ID does not exist" containerID="5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51"
Oct 30 21:57:05.535167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.535060    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51"} err="failed to get container status \"5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51\": rpc error: code = NotFound desc = could not find container \"5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51\": container with ID starting with 5d9e7ef7bf8ca62758ffe9029cfb9d3328591ceffc91959cbfad147771f61f51 not found: ID does not exist"
Oct 30 21:57:05.535167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.535077    4549 scope.go:122] "RemoveContainer" containerID="b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143"
Oct 30 21:57:05.543913 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.543890    4549 scope.go:122] "RemoveContainer" containerID="b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143"
Oct 30 21:57:05.544220 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:05.544201    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143\": container with ID starting with b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143 not found: ID does not exist" containerID="b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143"
Oct 30 21:57:05.544294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.544226    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143"} err="failed to get container status \"b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143\": rpc error: code = NotFound desc = could not find container \"b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143\": container with ID starting with b13a18f3edd1bd14dfb840e18f76f5d3da142c544a0f481dd60ac1b758efc143 not found: ID does not exist"
Oct 30 21:57:05.596299 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.596257    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"]
Oct 30 21:57:05.604255 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.604193    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc"]
Oct 30 21:57:05.666086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.666061    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"]
Oct 30 21:57:05.689244 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.689217    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s"]
Oct 30 21:57:05.787801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:05.787768    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:06.342031 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.342002    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:57:06.396969 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.396876    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.396968    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397049    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397127    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397156    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397179    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.397334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397197    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397225    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397244    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397334 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397290    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz\") pod \"71f46951-9378-4dee-9f2e-42d72da5ee1f\" (UID: \"71f46951-9378-4dee-9f2e-42d72da5ee1f\") "
Oct 30 21:57:06.397693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397394    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.397693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397627    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.397693 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397648    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.398017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.397991    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:06.398391 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.398340    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.398510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.398403    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.400062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.400033    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.400062 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.400053    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:06.400859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.400798    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "kube-api-access-9vswz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:06.404960 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.404932    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo" pod "71f46951-9378-4dee-9f2e-42d72da5ee1f" (UID: "71f46951-9378-4dee-9f2e-42d72da5ee1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:57:06.483916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.483879    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"]
Oct 30 21:57:06.499317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499287    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.499317 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499315    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.499535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499329    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.499535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499341    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f46951-9378-4dee-9f2e-42d72da5ee1f-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.499535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499351    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f46951-9378-4dee-9f2e-42d72da5ee1f-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.499535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499365    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f46951-9378-4dee-9f2e-42d72da5ee1f-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.499535 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.499375    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vswz\" (UniqueName: \"kubernetes.io/projected/71f46951-9378-4dee-9f2e-42d72da5ee1f-kube-api-access-9vswz\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:06.535318 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.535284    4549 generic.go:356] "Generic (PLEG): container finished" podID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerID="f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a" exitCode=0
Oct 30 21:57:06.535545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.535341    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"
Oct 30 21:57:06.535545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.535364    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" event={"ID":"71f46951-9378-4dee-9f2e-42d72da5ee1f","Type":"ContainerDied","Data":"f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a"}
Oct 30 21:57:06.535545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.535436    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn" event={"ID":"71f46951-9378-4dee-9f2e-42d72da5ee1f","Type":"ContainerDied","Data":"d09577ee5359e6c5f6c6333d238c66ae9af619bf60d0652c3b9ca9b631d08b9c"}
Oct 30 21:57:06.535545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.535466    4549 scope.go:122] "RemoveContainer" containerID="f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a"
Oct 30 21:57:06.548919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.548891    4549 scope.go:122] "RemoveContainer" containerID="e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"
Oct 30 21:57:06.558722 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.558704    4549 scope.go:122] "RemoveContainer" containerID="f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a"
Oct 30 21:57:06.559055 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:06.559034    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a\": container with ID starting with f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a not found: ID does not exist" containerID="f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a"
Oct 30 21:57:06.559140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.559066    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a"} err="failed to get container status \"f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a\": rpc error: code = NotFound desc = could not find container \"f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a\": container with ID starting with f1cf1359ebfe251debc5d9d122bfa401a74611034388f04aeb31c461781db64a not found: ID does not exist"
Oct 30 21:57:06.559140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.559083    4549 scope.go:122] "RemoveContainer" containerID="e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"
Oct 30 21:57:06.559337 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:06.559317    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510\": container with ID starting with e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510 not found: ID does not exist" containerID="e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"
Oct 30 21:57:06.559408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.559342    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510"} err="failed to get container status \"e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510\": rpc error: code = NotFound desc = could not find container \"e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510\": container with ID starting with e38fca479630307809615bbd329359cdb7a72fb42350fe32e21a3fdf51f63510 not found: ID does not exist"
Oct 30 21:57:06.594069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.594035    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"]
Oct 30 21:57:06.600563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.600517    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn"]
Oct 30 21:57:06.645253 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:06.645220    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:07.059155 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:07.054056    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:07.342991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:07.342954    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" path="/var/lib/kubelet/pods/71f46951-9378-4dee-9f2e-42d72da5ee1f/volumes"
Oct 30 21:57:07.343435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:07.343365    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b469b4b0-6207-42a6-b4e1-a89b457c975f" path="/var/lib/kubelet/pods/b469b4b0-6207-42a6-b4e1-a89b457c975f/volumes"
Oct 30 21:57:07.343705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:07.343682    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b70988b9-c257-47f2-98de-5eac5f217367" path="/var/lib/kubelet/pods/b70988b9-c257-47f2-98de-5eac5f217367/volumes"
Oct 30 21:57:08.538412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:08.538377    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:09.082418 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.082383    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:57:09.082609 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.082441    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:57:09.568866 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.568817    4549 generic.go:356] "Generic (PLEG): container finished" podID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerID="c5df2e9b9bde63f28ea45fc6974a67f4f542529ff1932b54d36f6d2e68759aa5" exitCode=0
Oct 30 21:57:09.569345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.568865    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" event={"ID":"32c9470c-c9cb-41f2-be69-23b1685f1191","Type":"ContainerDied","Data":"c5df2e9b9bde63f28ea45fc6974a67f4f542529ff1932b54d36f6d2e68759aa5"}
Oct 30 21:57:09.569345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.568943    4549 scope.go:122] "RemoveContainer" containerID="37da3dcdbf8df87074444099ce7b325f7f06118ff59f3376d60bb7f7826df798"
Oct 30 21:57:09.635801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.635774    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:57:09.735419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735333    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735386    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735419 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735409    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735441    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735467    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735561    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735640    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.735725 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735690    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:09.736002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735697    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.736002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735803    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token\") pod \"32c9470c-c9cb-41f2-be69-23b1685f1191\" (UID: \"32c9470c-c9cb-41f2-be69-23b1685f1191\") "
Oct 30 21:57:09.736002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735910    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:09.736002 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.735918    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:09.736193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.736140    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:09.736242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.736203    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:09.736291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.736266    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.736291 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.736283    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.736360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.736298    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c9470c-c9cb-41f2-be69-23b1685f1191-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.736360 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.736310    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.738966 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.738939    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:09.739086 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.738939    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:57:09.741948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.741925    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "kube-api-access-pkkgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:09.741948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.741938    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token" pod "32c9470c-c9cb-41f2-be69-23b1685f1191" (UID: "32c9470c-c9cb-41f2-be69-23b1685f1191"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:09.837041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.836994    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkkgn\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-kube-api-access-pkkgn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.837041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.837027    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.837041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.837037    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.837041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.837045    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:09.837041 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:09.837055    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c9470c-c9cb-41f2-be69-23b1685f1191-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:10.173614 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:10.173569    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:10.574995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:10.574968    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"
Oct 30 21:57:10.574995 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:10.574985    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz" event={"ID":"32c9470c-c9cb-41f2-be69-23b1685f1191","Type":"ContainerDied","Data":"f98689de6c6cddb9692fa31904d7f15bd8073021fa246e5daa33ab04584293c8"}
Oct 30 21:57:10.575512 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:10.575046    4549 scope.go:122] "RemoveContainer" containerID="c5df2e9b9bde63f28ea45fc6974a67f4f542529ff1932b54d36f6d2e68759aa5"
Oct 30 21:57:10.652038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:10.652000    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"]
Oct 30 21:57:10.663126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:10.663097    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz"]
Oct 30 21:57:11.342630 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:11.342590    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" path="/var/lib/kubelet/pods/32c9470c-c9cb-41f2-be69-23b1685f1191/volumes"
Oct 30 21:57:11.444172 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:11.444141    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:12.240324 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:12.240289    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:12.423540 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:12.423466    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:13.370600 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:13.369234    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:13.609074 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:13.609042    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.119137 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:14.119100    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.373450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:14.373367    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:16.064362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:16.064326    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:16.218017 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:16.217974    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:16.472936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:16.471631    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:18.267301 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:18.267258    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:19.721677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:19.721646    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.214193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.214155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.505043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.504952    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main" containerID="cri-o://6164741282f6ffc1f8cce2ffbd2d35daf1308ccb26efcbdcbdf9b5a4f1bf334c" gracePeriod=45
Oct 30 21:57:21.637980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.637947    4549 generic.go:356] "Generic (PLEG): container finished" podID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerID="6164741282f6ffc1f8cce2ffbd2d35daf1308ccb26efcbdcbdf9b5a4f1bf334c" exitCode=0
Oct 30 21:57:21.638162 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.638027    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" event={"ID":"c188c359-c3b2-4ba5-aef9-57044df0bb21","Type":"ContainerDied","Data":"6164741282f6ffc1f8cce2ffbd2d35daf1308ccb26efcbdcbdf9b5a4f1bf334c"}
Oct 30 21:57:21.793457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.793431    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:57:21.960238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960190    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache\") pod \"c188c359-c3b2-4ba5-aef9-57044df0bb21\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") "
Oct 30 21:57:21.960432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960261    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm\") pod \"c188c359-c3b2-4ba5-aef9-57044df0bb21\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") "
Oct 30 21:57:21.960432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960311    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir\") pod \"c188c359-c3b2-4ba5-aef9-57044df0bb21\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") "
Oct 30 21:57:21.960432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960355    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home\") pod \"c188c359-c3b2-4ba5-aef9-57044df0bb21\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") "
Oct 30 21:57:21.960432 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960386    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n\" (UniqueName: \"kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n\") pod \"c188c359-c3b2-4ba5-aef9-57044df0bb21\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") "
Oct 30 21:57:21.960658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960467    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs\" (UniqueName: \"kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs\") pod \"c188c359-c3b2-4ba5-aef9-57044df0bb21\" (UID: \"c188c359-c3b2-4ba5-aef9-57044df0bb21\") "
Oct 30 21:57:21.960658 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960497    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache" pod "c188c359-c3b2-4ba5-aef9-57044df0bb21" (UID: "c188c359-c3b2-4ba5-aef9-57044df0bb21"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:21.960782 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960653    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir" pod "c188c359-c3b2-4ba5-aef9-57044df0bb21" (UID: "c188c359-c3b2-4ba5-aef9-57044df0bb21"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:21.960965 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.960930    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home" pod "c188c359-c3b2-4ba5-aef9-57044df0bb21" (UID: "c188c359-c3b2-4ba5-aef9-57044df0bb21"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:21.961103 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.961081    4549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-tmp-dir\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:21.961179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.961105    4549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-home\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:21.961179 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.961119    4549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-model-cache\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:21.962930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.962902    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm" pod "c188c359-c3b2-4ba5-aef9-57044df0bb21" (UID: "c188c359-c3b2-4ba5-aef9-57044df0bb21"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:21.963037 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.962964    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n" pod "c188c359-c3b2-4ba5-aef9-57044df0bb21" (UID: "c188c359-c3b2-4ba5-aef9-57044df0bb21"). InnerVolumeSpecName "kube-api-access-wfx6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:21.963238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:21.963220    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs" pod "c188c359-c3b2-4ba5-aef9-57044df0bb21" (UID: "c188c359-c3b2-4ba5-aef9-57044df0bb21"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:22.062400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.062310    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfx6n\" (UniqueName: \"kubernetes.io/projected/c188c359-c3b2-4ba5-aef9-57044df0bb21-kube-api-access-wfx6n\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:22.062400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.062345    4549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c188c359-c3b2-4ba5-aef9-57044df0bb21-tls-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:22.062400 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.062355    4549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c188c359-c3b2-4ba5-aef9-57044df0bb21-dshm\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:22.645072 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.645038    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"
Oct 30 21:57:22.645570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.645041    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8" event={"ID":"c188c359-c3b2-4ba5-aef9-57044df0bb21","Type":"ContainerDied","Data":"307106f15a1ece397d46bc48762ce862044db7b61ebfbf2b342b3471e662568c"}
Oct 30 21:57:22.645570 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.645159    4549 scope.go:122] "RemoveContainer" containerID="6164741282f6ffc1f8cce2ffbd2d35daf1308ccb26efcbdcbdf9b5a4f1bf334c"
Oct 30 21:57:22.710183 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.710144    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"]
Oct 30 21:57:22.711880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.711854    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-839163/distinct-2-w4-839163-kserve-77c9dfc99d-8w8v8"]
Oct 30 21:57:22.766748 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:22.766709    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:23.343889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:23.343846    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" path="/var/lib/kubelet/pods/c188c359-c3b2-4ba5-aef9-57044df0bb21/volumes"
Oct 30 21:57:24.761362 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:24.761325    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.054569 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.054533    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"]
Oct 30 21:57:26.055135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.054815    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" podUID="59c7073f-2822-494d-8610-fc747fc2affe" containerName="maas-api" containerID="cri-o://1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e" gracePeriod=30
Oct 30 21:57:26.095042 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.095006    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" podUID="59c7073f-2822-494d-8610-fc747fc2affe" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.187:8443/health\": dial tcp 10.133.0.187:8443: connect: connection refused"
Oct 30 21:57:26.156170 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.156128    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.345361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.345334    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:57:26.422527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.422492    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"]
Oct 30 21:57:26.422759 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.422723    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj" podUID="18f23302-342c-41a8-b18c-68ec89497e8b" containerName="maas-api" containerID="cri-o://f80c13cb1c7b3ca0f4ae2ddac521e0babefc6bb0439da76bf52eaad64cfa4ac0" gracePeriod=30
Oct 30 21:57:26.508553 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.508518    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls\") pod \"59c7073f-2822-494d-8610-fc747fc2affe\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") "
Oct 30 21:57:26.508752 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.508700    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc\" (UniqueName: \"kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc\") pod \"59c7073f-2822-494d-8610-fc747fc2affe\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") "
Oct 30 21:57:26.508823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.508748    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs\") pod \"59c7073f-2822-494d-8610-fc747fc2affe\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") "
Oct 30 21:57:26.508932 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.508895    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca\") pod \"59c7073f-2822-494d-8610-fc747fc2affe\" (UID: \"59c7073f-2822-494d-8610-fc747fc2affe\") "
Oct 30 21:57:26.509283 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.509255    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca" pod "59c7073f-2822-494d-8610-fc747fc2affe" (UID: "59c7073f-2822-494d-8610-fc747fc2affe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:26.509578 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.509370    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c7073f-2822-494d-8610-fc747fc2affe-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.511163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.511131    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs" pod "59c7073f-2822-494d-8610-fc747fc2affe" (UID: "59c7073f-2822-494d-8610-fc747fc2affe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:26.511276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.511240    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls" pod "59c7073f-2822-494d-8610-fc747fc2affe" (UID: "59c7073f-2822-494d-8610-fc747fc2affe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:26.511276 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.511269    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc" pod "59c7073f-2822-494d-8610-fc747fc2affe" (UID: "59c7073f-2822-494d-8610-fc747fc2affe"). InnerVolumeSpecName "kube-api-access-kkwqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:26.516516 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.516481    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.579192 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:26.579070    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:57:26.611094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.611056    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkwqc\" (UniqueName: \"kubernetes.io/projected/59c7073f-2822-494d-8610-fc747fc2affe-kube-api-access-kkwqc\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.611094 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.611097    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.611358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.611114    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59c7073f-2822-494d-8610-fc747fc2affe-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.668947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.668759    4549 generic.go:356] "Generic (PLEG): container finished" podID="18f23302-342c-41a8-b18c-68ec89497e8b" containerID="f80c13cb1c7b3ca0f4ae2ddac521e0babefc6bb0439da76bf52eaad64cfa4ac0" exitCode=0
Oct 30 21:57:26.668947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.668913    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj" event={"ID":"18f23302-342c-41a8-b18c-68ec89497e8b","Type":"ContainerDied","Data":"f80c13cb1c7b3ca0f4ae2ddac521e0babefc6bb0439da76bf52eaad64cfa4ac0"}
Oct 30 21:57:26.669201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.668984    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj" event={"ID":"18f23302-342c-41a8-b18c-68ec89497e8b","Type":"ContainerDied","Data":"f3f8ce9becd9f724a74ec90532357d6c9f58e7d6d96752ddfa6bc3eb956e7dd9"}
Oct 30 21:57:26.669201 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.669010    4549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3f8ce9becd9f724a74ec90532357d6c9f58e7d6d96752ddfa6bc3eb956e7dd9"
Oct 30 21:57:26.675097 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.675050    4549 generic.go:356] "Generic (PLEG): container finished" podID="59c7073f-2822-494d-8610-fc747fc2affe" containerID="1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e" exitCode=0
Oct 30 21:57:26.675358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.675167    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" event={"ID":"59c7073f-2822-494d-8610-fc747fc2affe","Type":"ContainerDied","Data":"1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e"}
Oct 30 21:57:26.675358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.675225    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc" event={"ID":"59c7073f-2822-494d-8610-fc747fc2affe","Type":"ContainerDied","Data":"8136e30c24e57bfe974fd08abb569e89334c95e3d700cda3ba7d55f44e9de623"}
Oct 30 21:57:26.675358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.675258    4549 scope.go:122] "RemoveContainer" containerID="1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e"
Oct 30 21:57:26.675358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.675283    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"
Oct 30 21:57:26.681104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.680878    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:57:26.687119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.687086    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.689557 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.689532    4549 scope.go:122] "RemoveContainer" containerID="1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e"
Oct 30 21:57:26.689933 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:26.689910    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e\": container with ID starting with 1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e not found: ID does not exist" containerID="1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e"
Oct 30 21:57:26.690001 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.689947    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e"} err="failed to get container status \"1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e\": rpc error: code = NotFound desc = could not find container \"1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e\": container with ID starting with 1efa7f2352553c8ddb9c4b215b7c30655cfbe9f10ecc22681bcaf5b4e00ce18e not found: ID does not exist"
Oct 30 21:57:26.775129 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.775079    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"]
Oct 30 21:57:26.781670 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.781635    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-da52fd-87c5c9cd-b29bc"]
Oct 30 21:57:26.813726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.813686    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls\") pod \"18f23302-342c-41a8-b18c-68ec89497e8b\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") "
Oct 30 21:57:26.813936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.813794    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs\") pod \"18f23302-342c-41a8-b18c-68ec89497e8b\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") "
Oct 30 21:57:26.813936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.813873    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca\") pod \"18f23302-342c-41a8-b18c-68ec89497e8b\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") "
Oct 30 21:57:26.813936 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.813905    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc\" (UniqueName: \"kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc\") pod \"18f23302-342c-41a8-b18c-68ec89497e8b\" (UID: \"18f23302-342c-41a8-b18c-68ec89497e8b\") "
Oct 30 21:57:26.814275 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.814242    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca" pod "18f23302-342c-41a8-b18c-68ec89497e8b" (UID: "18f23302-342c-41a8-b18c-68ec89497e8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:26.816114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.816096    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls" pod "18f23302-342c-41a8-b18c-68ec89497e8b" (UID: "18f23302-342c-41a8-b18c-68ec89497e8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:26.816634 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.816609    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs" pod "18f23302-342c-41a8-b18c-68ec89497e8b" (UID: "18f23302-342c-41a8-b18c-68ec89497e8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:26.816701 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.816635    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc" pod "18f23302-342c-41a8-b18c-68ec89497e8b" (UID: "18f23302-342c-41a8-b18c-68ec89497e8b"). InnerVolumeSpecName "kube-api-access-qgjdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:26.911641 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.911551    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.923475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.915569    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.923475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.915606    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18f23302-342c-41a8-b18c-68ec89497e8b-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.923475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.915624    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18f23302-342c-41a8-b18c-68ec89497e8b-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:26.923475 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:26.915644    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgjdc\" (UniqueName: \"kubernetes.io/projected/18f23302-342c-41a8-b18c-68ec89497e8b-kube-api-access-qgjdc\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:27.154726 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.154690    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:27.329565 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.329528    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:27.343865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.343752    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59c7073f-2822-494d-8610-fc747fc2affe" path="/var/lib/kubelet/pods/59c7073f-2822-494d-8610-fc747fc2affe/volumes"
Oct 30 21:57:27.681950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.681863    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"
Oct 30 21:57:27.733188 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.733135    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"]
Oct 30 21:57:27.745045 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.745011    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-839163-674d48c657-vlmkj"]
Oct 30 21:57:27.779202 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:27.779170    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:29.038656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:29.038622    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:29.342947 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:29.342917    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18f23302-342c-41a8-b18c-68ec89497e8b" path="/var/lib/kubelet/pods/18f23302-342c-41a8-b18c-68ec89497e8b/volumes"
Oct 30 21:57:30.582865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:30.581667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:32.024633 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:32.024598    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.189370 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.189337    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"]
Oct 30 21:57:33.189954 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.189625    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7" podUID="e08d2214-1986-49ed-8b92-687522c66890" containerName="maas-api" containerID="cri-o://6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57" gracePeriod=30
Oct 30 21:57:33.315967 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.315934    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.362878 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.362808    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"]
Oct 30 21:57:33.363145 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.363102    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z" podUID="d1482a83-e8a0-40a9-86de-755c84032fc4" containerName="payload-processing" containerID="cri-o://f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490" gracePeriod=30
Oct 30 21:57:33.379186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.378189    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"]
Oct 30 21:57:33.379186 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.378589    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4" podUID="f0f51833-a02d-40c7-8067-7364fca1aff2" containerName="payload-pre-processing" containerID="cri-o://cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3" gracePeriod=30
Oct 30 21:57:33.474662 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.474627    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.525889 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.523043    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:57:33.583022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.582223    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca\") pod \"e08d2214-1986-49ed-8b92-687522c66890\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") "
Oct 30 21:57:33.583022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.582348    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls\") pod \"e08d2214-1986-49ed-8b92-687522c66890\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") "
Oct 30 21:57:33.583022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.582454    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb\" (UniqueName: \"kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb\") pod \"e08d2214-1986-49ed-8b92-687522c66890\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") "
Oct 30 21:57:33.583022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.582504    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs\") pod \"e08d2214-1986-49ed-8b92-687522c66890\" (UID: \"e08d2214-1986-49ed-8b92-687522c66890\") "
Oct 30 21:57:33.583022 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.582933    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca" pod "e08d2214-1986-49ed-8b92-687522c66890" (UID: "e08d2214-1986-49ed-8b92-687522c66890"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:33.585024 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.584990    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb" pod "e08d2214-1986-49ed-8b92-687522c66890" (UID: "e08d2214-1986-49ed-8b92-687522c66890"). InnerVolumeSpecName "kube-api-access-ctqvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:33.585163 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.585143    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls" pod "e08d2214-1986-49ed-8b92-687522c66890" (UID: "e08d2214-1986-49ed-8b92-687522c66890"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:33.586134 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.586105    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs" pod "e08d2214-1986-49ed-8b92-687522c66890" (UID: "e08d2214-1986-49ed-8b92-687522c66890"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:33.635571 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.635443    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.641872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.641820    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:57:33.670486 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.670456    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.683592    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c\" (UniqueName: \"kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c\") pod \"d1482a83-e8a0-40a9-86de-755c84032fc4\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") "
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.683651    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume\") pod \"d1482a83-e8a0-40a9-86de-755c84032fc4\" (UID: \"d1482a83-e8a0-40a9-86de-755c84032fc4\") "
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.684073    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctqvb\" (UniqueName: \"kubernetes.io/projected/e08d2214-1986-49ed-8b92-687522c66890-kube-api-access-ctqvb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.684102    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.684118    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e08d2214-1986-49ed-8b92-687522c66890-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.684132    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e08d2214-1986-49ed-8b92-687522c66890-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.684952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.684591    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume" pod "d1482a83-e8a0-40a9-86de-755c84032fc4" (UID: "d1482a83-e8a0-40a9-86de-755c84032fc4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:33.691980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.691943    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c" pod "d1482a83-e8a0-40a9-86de-755c84032fc4" (UID: "d1482a83-e8a0-40a9-86de-755c84032fc4"). InnerVolumeSpecName "kube-api-access-l5n2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:33.726911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.726757    4549 generic.go:356] "Generic (PLEG): container finished" podID="d1482a83-e8a0-40a9-86de-755c84032fc4" containerID="f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490" exitCode=0
Oct 30 21:57:33.726911 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.726815    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"
Oct 30 21:57:33.727198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.726855    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z" event={"ID":"d1482a83-e8a0-40a9-86de-755c84032fc4","Type":"ContainerDied","Data":"f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490"}
Oct 30 21:57:33.727198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.727003    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z" event={"ID":"d1482a83-e8a0-40a9-86de-755c84032fc4","Type":"ContainerDied","Data":"9f7648b1886a176ec95277ff698572fa2fcfc5131afd0da7d250c720bb7d156d"}
Oct 30 21:57:33.727198 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.727039    4549 scope.go:122] "RemoveContainer" containerID="f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490"
Oct 30 21:57:33.730541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.730513    4549 generic.go:356] "Generic (PLEG): container finished" podID="f0f51833-a02d-40c7-8067-7364fca1aff2" containerID="cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3" exitCode=0
Oct 30 21:57:33.730712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.730625    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4" event={"ID":"f0f51833-a02d-40c7-8067-7364fca1aff2","Type":"ContainerDied","Data":"cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3"}
Oct 30 21:57:33.730712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.730698    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4" event={"ID":"f0f51833-a02d-40c7-8067-7364fca1aff2","Type":"ContainerDied","Data":"7f4442e562f733d401d7f94afe8e9fd07729c6b4789e7af20514dbfcfbbff059"}
Oct 30 21:57:33.730931 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.730798    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"
Oct 30 21:57:33.734572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.732549    4549 generic.go:356] "Generic (PLEG): container finished" podID="e08d2214-1986-49ed-8b92-687522c66890" containerID="6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57" exitCode=0
Oct 30 21:57:33.734572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.732604    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7" event={"ID":"e08d2214-1986-49ed-8b92-687522c66890","Type":"ContainerDied","Data":"6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57"}
Oct 30 21:57:33.734572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.732646    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7" event={"ID":"e08d2214-1986-49ed-8b92-687522c66890","Type":"ContainerDied","Data":"088848be363c8355a79ce2ac41cdde0c7276c61e3254f782b282f9c6edbad1cf"}
Oct 30 21:57:33.734572 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.732722    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"
Oct 30 21:57:33.744527 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.744502    4549 scope.go:122] "RemoveContainer" containerID="f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490"
Oct 30 21:57:33.744994 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:33.744963    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490\": container with ID starting with f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490 not found: ID does not exist" containerID="f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490"
Oct 30 21:57:33.745119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.745007    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490"} err="failed to get container status \"f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490\": rpc error: code = NotFound desc = could not find container \"f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490\": container with ID starting with f227afc8a7f30541d3eec019ac6d4dbedea21826752a917f81046b11e3f37490 not found: ID does not exist"
Oct 30 21:57:33.745119 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.745032    4549 scope.go:122] "RemoveContainer" containerID="cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3"
Oct 30 21:57:33.761322 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.761292    4549 scope.go:122] "RemoveContainer" containerID="cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3"
Oct 30 21:57:33.765145 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:33.765108    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3\": container with ID starting with cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3 not found: ID does not exist" containerID="cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3"
Oct 30 21:57:33.765279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.765169    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3"} err="failed to get container status \"cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3\": rpc error: code = NotFound desc = could not find container \"cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3\": container with ID starting with cc2bd349621189319308195e9e7e070de427c998db2f1c16d7cdc3892225bdd3 not found: ID does not exist"
Oct 30 21:57:33.765279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.765196    4549 scope.go:122] "RemoveContainer" containerID="6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57"
Oct 30 21:57:33.779173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.779147    4549 scope.go:122] "RemoveContainer" containerID="6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57"
Oct 30 21:57:33.779465 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:33.779445    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57\": container with ID starting with 6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57 not found: ID does not exist" containerID="6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57"
Oct 30 21:57:33.779547 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.779473    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57"} err="failed to get container status \"6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57\": rpc error: code = NotFound desc = could not find container \"6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57\": container with ID starting with 6d56b0a73235a1c1d670df3a052ef5d512c15990d18cbb1c6982c85858584f57 not found: ID does not exist"
Oct 30 21:57:33.785073 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.785048    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume\") pod \"f0f51833-a02d-40c7-8067-7364fca1aff2\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") "
Oct 30 21:57:33.785223 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.785147    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql\" (UniqueName: \"kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql\") pod \"f0f51833-a02d-40c7-8067-7364fca1aff2\" (UID: \"f0f51833-a02d-40c7-8067-7364fca1aff2\") "
Oct 30 21:57:33.785423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.785407    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5n2c\" (UniqueName: \"kubernetes.io/projected/d1482a83-e8a0-40a9-86de-755c84032fc4-kube-api-access-l5n2c\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.785545 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.785426    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1482a83-e8a0-40a9-86de-755c84032fc4-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.785603 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.785552    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume" pod "f0f51833-a02d-40c7-8067-7364fca1aff2" (UID: "f0f51833-a02d-40c7-8067-7364fca1aff2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:33.787592 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.787560    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql" pod "f0f51833-a02d-40c7-8067-7364fca1aff2" (UID: "f0f51833-a02d-40c7-8067-7364fca1aff2"). InnerVolumeSpecName "kube-api-access-f7zql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:33.829015 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.828978    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"]
Oct 30 21:57:33.838482 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.838443    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z"]
Oct 30 21:57:33.850872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.850799    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.886298 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.886260    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7zql\" (UniqueName: \"kubernetes.io/projected/f0f51833-a02d-40c7-8067-7364fca1aff2-kube-api-access-f7zql\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.886561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.886536    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f51833-a02d-40c7-8067-7364fca1aff2-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:33.888595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.888569    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"]
Oct 30 21:57:33.898704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:33.898680    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-480d41-6879884bc6-2ntn7"]
Oct 30 21:57:34.091689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:34.091651    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"]
Oct 30 21:57:34.103654 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:34.103613    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4"]
Oct 30 21:57:34.114605 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:34.114564    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.545210 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:34.545179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.342510 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:35.342475    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1482a83-e8a0-40a9-86de-755c84032fc4" path="/var/lib/kubelet/pods/d1482a83-e8a0-40a9-86de-755c84032fc4/volumes"
Oct 30 21:57:35.342948 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:35.342934    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e08d2214-1986-49ed-8b92-687522c66890" path="/var/lib/kubelet/pods/e08d2214-1986-49ed-8b92-687522c66890/volumes"
Oct 30 21:57:35.343264 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:35.343252    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0f51833-a02d-40c7-8067-7364fca1aff2" path="/var/lib/kubelet/pods/f0f51833-a02d-40c7-8067-7364fca1aff2/volumes"
Oct 30 21:57:35.809567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:35.809483    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.060689 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:36.060600    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.516548 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:36.516513    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.774069 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:36.773970    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.011081 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:37.011046    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.177773 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:37.177742    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.571408 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:37.571373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:38.014327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:38.014233    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:39.551850 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:39.551776    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.103139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:41.103095    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.292743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:41.292702    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.413226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:41.413127    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"]
Oct 30 21:57:41.413421 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:41.413384    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy" containerID="cri-o://478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd" gracePeriod=30
Oct 30 21:57:41.525314 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:41.525282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.123075 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:42.123033    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:43.143754 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:43.143711    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.155671 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:44.155630    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.446513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:44.446422    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.636919 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:44.636884    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.816169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:44.816134    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:45.070778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:45.070682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:46.298460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.298425    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:46.676039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.676011    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:57:46.803448 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.803407    4549 generic.go:356] "Generic (PLEG): container finished" podID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerID="478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd" exitCode=0
Oct 30 21:57:46.803637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.803458    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"
Oct 30 21:57:46.803637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.803479    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" event={"ID":"e8adf648-d12f-4e04-95be-2b76723a4b97","Type":"ContainerDied","Data":"478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd"}
Oct 30 21:57:46.803637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.803542    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf" event={"ID":"e8adf648-d12f-4e04-95be-2b76723a4b97","Type":"ContainerDied","Data":"2f15c222b35bc3a480cfbcfedc12d0b0290030c46df2f95210b9df783b9222fa"}
Oct 30 21:57:46.803637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.803565    4549 scope.go:122] "RemoveContainer" containerID="478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd"
Oct 30 21:57:46.813279 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.813252    4549 scope.go:122] "RemoveContainer" containerID="6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"
Oct 30 21:57:46.822159 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822126    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822295 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822227    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822303    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822343    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822386    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:46.822522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822434    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822474    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822522 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822511    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822547    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822540    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:46.822916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822588    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo\") pod \"e8adf648-d12f-4e04-95be-2b76723a4b97\" (UID: \"e8adf648-d12f-4e04-95be-2b76723a4b97\") "
Oct 30 21:57:46.822916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822605    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:46.822916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822644    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:46.822916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.822802    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:46.823242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823008    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.823242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823028    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.823242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823042    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8adf648-d12f-4e04-95be-2b76723a4b97-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.823242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823055    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.823242 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823070    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.823563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823495    4549 scope.go:122] "RemoveContainer" containerID="478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd"
Oct 30 21:57:46.823971 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:46.823945    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd\": container with ID starting with 478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd not found: ID does not exist" containerID="478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd"
Oct 30 21:57:46.824089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823977    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd"} err="failed to get container status \"478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd\": rpc error: code = NotFound desc = could not find container \"478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd\": container with ID starting with 478f388fecde343c71d46c5975d89c30308c64a6b5bc7d54ad6e3e37b9fe56bd not found: ID does not exist"
Oct 30 21:57:46.824089 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.823996    4549 scope.go:122] "RemoveContainer" containerID="6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"
Oct 30 21:57:46.824271 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:46.824253    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd\": container with ID starting with 6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd not found: ID does not exist" containerID="6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"
Oct 30 21:57:46.824343 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.824279    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd"} err="failed to get container status \"6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd\": rpc error: code = NotFound desc = could not find container \"6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd\": container with ID starting with 6169460f1e80ca6c072e7c3f11be38d13fd425ef920468e8d58cd8c11f8beecd not found: ID does not exist"
Oct 30 21:57:46.824961 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.824935    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:46.825096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.825050    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "kube-api-access-48p52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:46.825287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.825267    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:46.825287 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.825279    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo" pod "e8adf648-d12f-4e04-95be-2b76723a4b97" (UID: "e8adf648-d12f-4e04-95be-2b76723a4b97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:57:46.845714 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.845642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:46.924118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.924085    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.924118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.924113    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48p52\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-kube-api-access-48p52\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.924118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.924123    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:46.924387 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:46.924132    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8adf648-d12f-4e04-95be-2b76723a4b97-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:57:47.169098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:47.169062    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"]
Oct 30 21:57:47.197110 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:47.197080    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf"]
Oct 30 21:57:47.342272 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:47.342235    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" path="/var/lib/kubelet/pods/e8adf648-d12f-4e04-95be-2b76723a4b97/volumes"
Oct 30 21:57:47.842457 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:47.842411    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:47.995385 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:47.995349    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:48.649687 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:48.649655    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:50.129399 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:50.129365    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:51.618304 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:51.618266    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:52.672560 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:52.672528    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:52.820423 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:52.820390    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:53.001021 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:53.000939    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:53.210229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:53.210198    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:54.943392 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:54.943357    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:56.414613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:56.414489    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:56.573362 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:57:56.573325    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:57:57.901680 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:57.901642    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:58.406891 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:58.406808    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:58.570899 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:58.570857    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:59.068494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:57:59.068448    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:00.266020 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:00.265970    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:01.952974 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:01.952939    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:03.428922 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:03.428874    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:04.859004 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:04.858956    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:05.810916 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:05.810864    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:05.976706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:05.976661    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:06.134525 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:06.134492    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"]
Oct 30 21:58:06.134783 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:06.134759    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" containerID="cri-o://964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9" gracePeriod=30
Oct 30 21:58:06.181259 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:06.181216    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:06.445168 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:06.445001    4549 patch_prober.go:29] interesting pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:58:06.445562 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:06.445527    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:58:06.450872 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:06.450807    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:08.029914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:08.029880    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:09.646218 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:09.646179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.003132 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.003095    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.401604 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.401577    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:58:11.452577 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.452544    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.460713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460685    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.460904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460780    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.460904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460811    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.460904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460863    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.460904 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460903    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.461150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460951    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.461150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.460979    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.461150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461014    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:11.461150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461029    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.461150 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461113    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h\") pod \"eba83462-ebf4-472f-8e5c-cb7b42c25321\" (UID: \"eba83462-ebf4-472f-8e5c-cb7b42c25321\") "
Oct 30 21:58:11.461429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461237    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:11.461429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461251    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:11.461429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461307    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:11.461619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461492    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba83462-ebf4-472f-8e5c-cb7b42c25321-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.461619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461511    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.461619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461524    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.461619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461537    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.461619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.461491    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:11.463474 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.463454    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "kube-api-access-5cs6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:11.463678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.463655    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:11.463678 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.463671    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:58:11.463823 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.463759    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token" pod "eba83462-ebf4-472f-8e5c-cb7b42c25321" (UID: "eba83462-ebf4-472f-8e5c-cb7b42c25321"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:11.569230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.569193    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.569230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.569231    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.569503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.569246    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.569503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.569260    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cs6h\" (UniqueName: \"kubernetes.io/projected/eba83462-ebf4-472f-8e5c-cb7b42c25321-kube-api-access-5cs6h\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.569503 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.569275    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba83462-ebf4-472f-8e5c-cb7b42c25321-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:11.599666 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.599621    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.921501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.921403    4549 generic.go:356] "Generic (PLEG): container finished" podID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerID="964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9" exitCode=0
Oct 30 21:58:11.921501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.921464    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"
Oct 30 21:58:11.921501 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.921483    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" event={"ID":"eba83462-ebf4-472f-8e5c-cb7b42c25321","Type":"ContainerDied","Data":"964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9"}
Oct 30 21:58:11.921790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.921542    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9" event={"ID":"eba83462-ebf4-472f-8e5c-cb7b42c25321","Type":"ContainerDied","Data":"8f89c42350da9a3d6315da4d334dea71e7446a5fffa08f05625311980d5d99ac"}
Oct 30 21:58:11.921790 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.921572    4549 scope.go:122] "RemoveContainer" containerID="964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9"
Oct 30 21:58:11.932043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.932022    4549 scope.go:122] "RemoveContainer" containerID="1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"
Oct 30 21:58:11.940494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.940472    4549 scope.go:122] "RemoveContainer" containerID="964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9"
Oct 30 21:58:11.941046 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:58:11.941024    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9\": container with ID starting with 964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9 not found: ID does not exist" containerID="964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9"
Oct 30 21:58:11.941139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.941057    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9"} err="failed to get container status \"964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9\": rpc error: code = NotFound desc = could not find container \"964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9\": container with ID starting with 964b836c1f5011f3ae71766d2fc74b5cc1d27500164635e56f64a1f7c3918dc9 not found: ID does not exist"
Oct 30 21:58:11.941139 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.941083    4549 scope.go:122] "RemoveContainer" containerID="1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"
Oct 30 21:58:11.941368 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:58:11.941347    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9\": container with ID starting with 1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9 not found: ID does not exist" containerID="1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"
Oct 30 21:58:11.941410 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.941375    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9"} err="failed to get container status \"1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9\": rpc error: code = NotFound desc = could not find container \"1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9\": container with ID starting with 1f7f969427f58a916b52b7ead791fde61af915b642289cf5b7b78461034ef8d9 not found: ID does not exist"
Oct 30 21:58:11.992743 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:11.992705    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"]
Oct 30 21:58:12.007434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:12.007394    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9"]
Oct 30 21:58:12.077558 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:12.077523    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:13.146104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:13.144904    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:13.342625 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:13.342592    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" path="/var/lib/kubelet/pods/eba83462-ebf4-472f-8e5c-cb7b42c25321/volumes"
Oct 30 21:58:14.611706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:14.611668    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:16.122384 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:16.122348    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.319114 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:17.319080    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.492138 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:17.492101    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:18.037136 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:18.037101    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:18.492451 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:18.492413    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:18.656228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:18.656195    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:18.882108 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:18.882076    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:19.103371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:19.103331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:19.253190 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:19.252801    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:19.393530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:19.393492    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:19.768241 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:19.768198    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:20.577228 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:20.577199    4549 scope.go:122] "RemoveContainer" containerID="f80c13cb1c7b3ca0f4ae2ddac521e0babefc6bb0439da76bf52eaad64cfa4ac0"
Oct 30 21:58:20.785898 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:20.785846    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:20.923361 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:20.923264    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.126412 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:21.126380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.248758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:21.248664    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.394013 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:21.393974    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.537402 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:21.537304    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:22.631225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:22.631191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:24.124766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:24.124714    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:25.497695 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:25.497656    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:25.634766 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:25.634720    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:25.779727 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:25.779640    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"]
Oct 30 21:58:25.779942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:25.779907    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy" containerID="cri-o://4cb77a12939b4548a98d5b7f3386c6503eb4edc851b29ac9678434ffe6186a89" gracePeriod=30
Oct 30 21:58:25.786327 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:25.786300    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:25.975294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:25.975256    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:26.411520 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:26.411488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:26.574936 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:58:26.574899    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:58:27.595788 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:27.595756    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:27.858697 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:27.858567    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:29.030064 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:29.029985    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:30.430541 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:30.430499    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:31.013345 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.013310    4549 generic.go:356] "Generic (PLEG): container finished" podID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerID="4cb77a12939b4548a98d5b7f3386c6503eb4edc851b29ac9678434ffe6186a89" exitCode=0
Oct 30 21:58:31.013563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.013384    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" event={"ID":"0080a530-5406-4ecb-8fcc-ad4fa5fb0793","Type":"ContainerDied","Data":"4cb77a12939b4548a98d5b7f3386c6503eb4edc851b29ac9678434ffe6186a89"}
Oct 30 21:58:31.013563 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.013471    4549 scope.go:122] "RemoveContainer" containerID="0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0"
Oct 30 21:58:31.033759 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:58:31.033726    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0\": container with ID starting with 0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0 not found: ID does not exist" containerID="0551dbc23f58829953b30d38c88b561625b4dcc882258df7b6d153c8435fbcb0"
Oct 30 21:58:31.037229 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.037210    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:58:31.156195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156150    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156195 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156201    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156231    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156442 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156363    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156515 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156477    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156561 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156530    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156556    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:31.156620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156563    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156620 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156596    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:31.156798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156622    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156798 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156653    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket\") pod \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\" (UID: \"0080a530-5406-4ecb-8fcc-ad4fa5fb0793\") "
Oct 30 21:58:31.156963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156806    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:31.156963 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.156920    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:31.157091 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.157067    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:31.157413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.157347    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.157413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.157377    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.157413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.157391    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.157413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.157405    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.157413 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.157420    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.158713 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.158685    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:58:31.158853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.158726    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:31.158853 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.158759    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:31.158943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.158913    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp" pod "0080a530-5406-4ecb-8fcc-ad4fa5fb0793" (UID: "0080a530-5406-4ecb-8fcc-ad4fa5fb0793"). InnerVolumeSpecName "kube-api-access-jbmpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:31.258357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.258318    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.258357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.258350    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbmpp\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-kube-api-access-jbmpp\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.258357 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.258360    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.258627 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.258378    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0080a530-5406-4ecb-8fcc-ad4fa5fb0793-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:31.428104 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.428013    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:31.927712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:31.927667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:32.019429 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:32.019399    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"
Oct 30 21:58:32.019656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:32.019443    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9" event={"ID":"0080a530-5406-4ecb-8fcc-ad4fa5fb0793","Type":"ContainerDied","Data":"df0a68419ec911f622a463d59e285b27a9021935b4fb1bb1fc8810dde59ecd52"}
Oct 30 21:58:32.019656 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:32.019512    4549 scope.go:122] "RemoveContainer" containerID="4cb77a12939b4548a98d5b7f3386c6503eb4edc851b29ac9678434ffe6186a89"
Oct 30 21:58:32.069530 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:32.069491    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"]
Oct 30 21:58:32.081358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:32.081322    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9"]
Oct 30 21:58:33.341637 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:33.341595    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" path="/var/lib/kubelet/pods/0080a530-5406-4ecb-8fcc-ad4fa5fb0793/volumes"
Oct 30 21:58:33.439461 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:33.439431    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:34.826875 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:34.826815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:36.210818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:36.210781    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:37.119128 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:37.119098    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:37.258225 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:37.258169    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:52.570779 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.570732    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"]
Oct 30 21:58:52.571381 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.571068    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n" podUID="24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" containerName="maas-api" containerID="cri-o://e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d" gracePeriod=30
Oct 30 21:58:52.653691 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.653656    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:52.829567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.829497    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:58:52.964867 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.964799    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls\") pod \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") "
Oct 30 21:58:52.965058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.964914    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j\" (UniqueName: \"kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j\") pod \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") "
Oct 30 21:58:52.965058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.964943    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs\") pod \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") "
Oct 30 21:58:52.965058 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.965032    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca\") pod \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\" (UID: \"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7\") "
Oct 30 21:58:52.965466 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.965436    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca" pod "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" (UID: "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:52.967366 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.967336    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls" pod "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" (UID: "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:52.967489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.967370    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j" pod "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" (UID: "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7"). InnerVolumeSpecName "kube-api-access-rxd9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:52.967489 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:52.967475    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs" pod "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" (UID: "24c47c92-cdde-40e5-bf9c-7a8762f7fdb7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:53.065434 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.065391    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:53.065677 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.065657    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:53.065741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.065681    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:53.065741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.065690    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxd9j\" (UniqueName: \"kubernetes.io/projected/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-kube-api-access-rxd9j\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:53.065741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.065699    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:58:53.114613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.114530    4549 generic.go:356] "Generic (PLEG): container finished" podID="24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" containerID="e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d" exitCode=0
Oct 30 21:58:53.114613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.114583    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"
Oct 30 21:58:53.114613 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.114594    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n" event={"ID":"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7","Type":"ContainerDied","Data":"e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d"}
Oct 30 21:58:53.114943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.114636    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n" event={"ID":"24c47c92-cdde-40e5-bf9c-7a8762f7fdb7","Type":"ContainerDied","Data":"f2d387eacd9210ec3f8c09fe7b1433ee97f0b7048511aee749680b1c05cd7191"}
Oct 30 21:58:53.114943 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.114658    4549 scope.go:122] "RemoveContainer" containerID="e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d"
Oct 30 21:58:53.123930 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.123909    4549 scope.go:122] "RemoveContainer" containerID="e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d"
Oct 30 21:58:53.124282 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:58:53.124258    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d\": container with ID starting with e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d not found: ID does not exist" containerID="e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d"
Oct 30 21:58:53.124367 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.124291    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d"} err="failed to get container status \"e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d\": rpc error: code = NotFound desc = could not find container \"e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d\": container with ID starting with e4260ba773ab7895d59f175f9940c20c0304e0003db49fdfdddd5dea50e7e43d not found: ID does not exist"
Oct 30 21:58:53.164989 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.164949    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"]
Oct 30 21:58:53.177169 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.177134    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ed088d-67dd7cdc98-tz95n"]
Oct 30 21:58:53.190803 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.190758    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:53.342209 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.342163    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" path="/var/lib/kubelet/pods/24c47c92-cdde-40e5-bf9c-7a8762f7fdb7/volumes"
Oct 30 21:58:53.660528 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:53.660491    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:56.572820 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:58:56.572785    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:58:59.682801 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:59.682760    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:59.812105 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:58:59.812062    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:00.121123 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:00.121081    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:04.770450 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:04.770407    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-q8pnp"]
Oct 30 21:59:04.771038 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:04.770662    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" podUID="2b78b21f-3a4b-4988-84de-851f172a5d9f" containerName="manager" containerID="cri-o://8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a" gracePeriod=30
Oct 30 21:59:04.821753 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:04.821712    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" podUID="2b78b21f-3a4b-4988-84de-851f172a5d9f" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.27:8081/readyz\": dial tcp 10.133.0.27:8081: connect: connection refused"
Oct 30 21:59:05.014029 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.013994    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:59:05.072710 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.072673    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8\" (UniqueName: \"kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8\") pod \"2b78b21f-3a4b-4988-84de-851f172a5d9f\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") "
Oct 30 21:59:05.072942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.072759    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs\") pod \"2b78b21f-3a4b-4988-84de-851f172a5d9f\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") "
Oct 30 21:59:05.072942 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.072857    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert\") pod \"2b78b21f-3a4b-4988-84de-851f172a5d9f\" (UID: \"2b78b21f-3a4b-4988-84de-851f172a5d9f\") "
Oct 30 21:59:05.075619 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.075583    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8" pod "2b78b21f-3a4b-4988-84de-851f172a5d9f" (UID: "2b78b21f-3a4b-4988-84de-851f172a5d9f"). InnerVolumeSpecName "kube-api-access-kwgf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:05.075758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.075712    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert" pod "2b78b21f-3a4b-4988-84de-851f172a5d9f" (UID: "2b78b21f-3a4b-4988-84de-851f172a5d9f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:05.075758 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.075730    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs" pod "2b78b21f-3a4b-4988-84de-851f172a5d9f" (UID: "2b78b21f-3a4b-4988-84de-851f172a5d9f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:05.174043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.174004    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwgf8\" (UniqueName: \"kubernetes.io/projected/2b78b21f-3a4b-4988-84de-851f172a5d9f-kube-api-access-kwgf8\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:59:05.174043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.174035    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:59:05.174043 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.174045    4549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b78b21f-3a4b-4988-84de-851f172a5d9f-webhook-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:59:05.175135 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.175112    4549 generic.go:356] "Generic (PLEG): container finished" podID="2b78b21f-3a4b-4988-84de-851f172a5d9f" containerID="8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a" exitCode=0
Oct 30 21:59:05.175238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.175176    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-q8pnp"
Oct 30 21:59:05.175238 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.175190    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" event={"ID":"2b78b21f-3a4b-4988-84de-851f172a5d9f","Type":"ContainerDied","Data":"8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a"}
Oct 30 21:59:05.175342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.175243    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-q8pnp" event={"ID":"2b78b21f-3a4b-4988-84de-851f172a5d9f","Type":"ContainerDied","Data":"4e2d2f2cc90c9457f2ca4bf5bbe704eac3fad75e8189f0ebd46b6d03996258ee"}
Oct 30 21:59:05.175342 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.175265    4549 scope.go:122] "RemoveContainer" containerID="8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a"
Oct 30 21:59:05.193772 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.193747    4549 scope.go:122] "RemoveContainer" containerID="8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a"
Oct 30 21:59:05.194170 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:59:05.194144    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a\": container with ID starting with 8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a not found: ID does not exist" containerID="8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a"
Oct 30 21:59:05.194226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.194186    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a"} err="failed to get container status \"8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a\": rpc error: code = NotFound desc = could not find container \"8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a\": container with ID starting with 8fec4127229ee35c13dae537f6e0d4a47e31fec5537616ad8e1019e777decc6a not found: ID does not exist"
Oct 30 21:59:05.229178 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.229140    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-q8pnp"]
Oct 30 21:59:05.233193 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.233164    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-q8pnp"]
Oct 30 21:59:05.343494 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.343419    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b78b21f-3a4b-4988-84de-851f172a5d9f" path="/var/lib/kubelet/pods/2b78b21f-3a4b-4988-84de-851f172a5d9f/volumes"
Oct 30 21:59:05.883744 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.883708    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-654c5dd44b-bn92k"]
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884056    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerName="payload-pre-processing"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884067    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerName="payload-pre-processing"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884081    4549 cpu_manager.go:414] "containerMap: removing container" podUID="2b78b21f-3a4b-4988-84de-851f172a5d9f" containerName="manager"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884086    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b78b21f-3a4b-4988-84de-851f172a5d9f" containerName="manager"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884093    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884097    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884104    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b5898524-e326-4e39-bdce-3aaff9b2b5b9" containerName="payload-pre-processing"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884110    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5898524-e326-4e39-bdce-3aaff9b2b5b9" containerName="payload-pre-processing"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884120    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e08d2214-1986-49ed-8b92-687522c66890" containerName="maas-api"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884126    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e08d2214-1986-49ed-8b92-687522c66890" containerName="maas-api"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884134    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884138    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884146    4549 cpu_manager.go:414] "containerMap: removing container" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884150    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884157    4549 cpu_manager.go:414] "containerMap: removing container" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884161    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884168    4549 cpu_manager.go:414] "containerMap: removing container" podUID="d1482a83-e8a0-40a9-86de-755c84032fc4" containerName="payload-processing"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884172    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1482a83-e8a0-40a9-86de-755c84032fc4" containerName="payload-processing"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884178    4549 cpu_manager.go:414] "containerMap: removing container" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy"
Oct 30 21:59:05.884177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884183    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884191    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884195    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884201    4549 cpu_manager.go:414] "containerMap: removing container" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884206    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884211    4549 cpu_manager.go:414] "containerMap: removing container" podUID="59724577-6c37-4b72-a404-725cb32ff7b3" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884217    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59724577-6c37-4b72-a404-725cb32ff7b3" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884224    4549 cpu_manager.go:414] "containerMap: removing container" podUID="18f23302-342c-41a8-b18c-68ec89497e8b" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884228    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18f23302-342c-41a8-b18c-68ec89497e8b" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884237    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b70988b9-c257-47f2-98de-5eac5f217367" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884241    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b70988b9-c257-47f2-98de-5eac5f217367" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884248    4549 cpu_manager.go:414] "containerMap: removing container" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884253    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884259    4549 cpu_manager.go:414] "containerMap: removing container" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884263    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884269    4549 cpu_manager.go:414] "containerMap: removing container" podUID="59c7073f-2822-494d-8610-fc747fc2affe" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884273    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59c7073f-2822-494d-8610-fc747fc2affe" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884279    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884283    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884289    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884293    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884299    4549 cpu_manager.go:414] "containerMap: removing container" podUID="abe73d82-7ffe-44bb-a9fe-ece88010a356" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884303    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe73d82-7ffe-44bb-a9fe-ece88010a356" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884310    4549 cpu_manager.go:414] "containerMap: removing container" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884314    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884320    4549 cpu_manager.go:414] "containerMap: removing container" podUID="b469b4b0-6207-42a6-b4e1-a89b457c975f" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884324    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b469b4b0-6207-42a6-b4e1-a89b457c975f" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884332    4549 cpu_manager.go:414] "containerMap: removing container" podUID="f0f51833-a02d-40c7-8067-7364fca1aff2" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884336    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0f51833-a02d-40c7-8067-7364fca1aff2" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884343    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884348    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884358    4549 cpu_manager.go:414] "containerMap: removing container" podUID="6a448953-34f7-4a32-985c-242827d5d848" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884362    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a448953-34f7-4a32-985c-242827d5d848" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884367    4549 cpu_manager.go:414] "containerMap: removing container" podUID="eb7ca725-d313-455a-89dc-9da4580a8755" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884372    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb7ca725-d313-455a-89dc-9da4580a8755" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884377    4549 cpu_manager.go:414] "containerMap: removing container" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884381    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884389    4549 cpu_manager.go:414] "containerMap: removing container" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884393    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884399    4549 cpu_manager.go:414] "containerMap: removing container" podUID="24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884403    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884409    4549 cpu_manager.go:414] "containerMap: removing container" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884413    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884422    4549 cpu_manager.go:414] "containerMap: removing container" podUID="67acd37b-ca60-4237-bfd6-bb38757d71bc" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884426    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67acd37b-ca60-4237-bfd6-bb38757d71bc" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884433    4549 cpu_manager.go:414] "containerMap: removing container" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884437    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884443    4549 cpu_manager.go:414] "containerMap: removing container" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884447    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884452    4549 cpu_manager.go:414] "containerMap: removing container" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884456    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884522    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffda1237-0394-47d7-bcbb-fddae855ef6d" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884532    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884543    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a448953-34f7-4a32-985c-242827d5d848" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884551    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb7ca725-d313-455a-89dc-9da4580a8755" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884557    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884562    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="41cdfe49-fc8c-43a9-a7a5-3515e5626c6e" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884568    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="24c47c92-cdde-40e5-bf9c-7a8762f7fdb7" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884572    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="42172e4b-91d7-4bab-8aa8-c22212cdfaa7" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884578    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="67acd37b-ca60-4237-bfd6-bb38757d71bc" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884582    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="9335aaf8-01d8-4ef5-b737-ee7e2b0fc04d" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884587    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884591    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="eba83462-ebf4-472f-8e5c-cb7b42c25321" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884597    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b36205f5-889f-4f2d-b0c7-8940b127181f" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884601    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b78b21f-3a4b-4988-84de-851f172a5d9f" containerName="manager"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884606    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884611    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5898524-e326-4e39-bdce-3aaff9b2b5b9" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884616    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e08d2214-1986-49ed-8b92-687522c66890" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884621    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8adf648-d12f-4e04-95be-2b76723a4b97" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884627    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884632    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1482a83-e8a0-40a9-86de-755c84032fc4" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884636    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884642    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9ee38ff-372c-424c-892b-a3b4880e0c34" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884647    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb817463-80b7-427d-b20f-5a1c6914f87d" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884652    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="59724577-6c37-4b72-a404-725cb32ff7b3" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884658    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="18f23302-342c-41a8-b18c-68ec89497e8b" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884662    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b70988b9-c257-47f2-98de-5eac5f217367" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884668    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="c188c359-c3b2-4ba5-aef9-57044df0bb21" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884672    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="59c7073f-2822-494d-8610-fc747fc2affe" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884677    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b562bdeb-bf87-4c6f-b22e-34792cce8cc8" containerName="main"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884682    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b526a0b0-b173-4352-8c9e-e868c1416071" containerName="payload-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884686    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="abe73d82-7ffe-44bb-a9fe-ece88010a356" containerName="maas-api"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884691    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="32c9470c-c9cb-41f2-be69-23b1685f1191" containerName="istio-proxy"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884695    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="b469b4b0-6207-42a6-b4e1-a89b457c975f" containerName="payload-pre-processing"
Oct 30 21:59:05.884818 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.884700    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0f51833-a02d-40c7-8067-7364fca1aff2" containerName="payload-pre-processing"
Oct 30 21:59:05.889453 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.889435    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:05.909865 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.909822    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:59:05.918046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.918017    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-bn92k"]
Oct 30 21:59:05.946460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.946422    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mhc8s\""
Oct 30 21:59:05.946460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.946422    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:59:05.946460 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.946454    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:59:05.981120 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.981088    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwx4b\" (UniqueName: \"kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:05.981289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.981131    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:05.981289 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:05.981245    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.081808 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.081772    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.081990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.081851    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.081990 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.081905    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwx4b\" (UniqueName: \"kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.084580 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.084551    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.084737 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.084716    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.102245 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.102210    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwx4b\" (UniqueName: \"kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b\") pod \"maas-controller-654c5dd44b-bn92k\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") " pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.202436 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.202343    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:06.496719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:06.496626    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-bn92k"]
Oct 30 21:59:06.500080 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:59:06.500042    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9402ffdf_7a1a_453c_a560_982f6642fb5b.slice/crio-32b23b99baf9df089ce17146b86c6f3b02623c3f301982123bd34493ba8ae982 WatchSource:0}: Error finding container 32b23b99baf9df089ce17146b86c6f3b02623c3f301982123bd34493ba8ae982: Status 404 returned error can't find the container with id 32b23b99baf9df089ce17146b86c6f3b02623c3f301982123bd34493ba8ae982
Oct 30 21:59:07.188066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:07.187971    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-bn92k" event={"ID":"9402ffdf-7a1a-453c-a560-982f6642fb5b","Type":"ContainerStarted","Data":"aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76"}
Oct 30 21:59:07.188066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:07.188038    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:07.188066 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:07.188060    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-bn92k" event={"ID":"9402ffdf-7a1a-453c-a560-982f6642fb5b","Type":"ContainerStarted","Data":"32b23b99baf9df089ce17146b86c6f3b02623c3f301982123bd34493ba8ae982"}
Oct 30 21:59:07.228235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:07.228176    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-654c5dd44b-bn92k" podStartSLOduration=1.886666476 podStartE2EDuration="2.228161298s" totalImagesPullingTime="341.494822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:07.226756106 +0000 UTC m=+3530.001156039" watchObservedRunningTime="2026-10-30 21:59:07.228161298 +0000 UTC m=+3530.002561252"
Oct 30 21:59:08.769795 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:08.769755    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:08.900350 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:08.900312    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:09.010599 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:09.010563    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:11.670706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:11.670667    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:14.554477 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:14.554446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:18.195991 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:18.195953    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:26.578145 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:59:26.578096    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 21:59:37.829096 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:37.829060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:37.947003 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:37.946956    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-bn92k"]
Oct 30 21:59:37.947235 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:37.947203    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-654c5dd44b-bn92k" podUID="9402ffdf-7a1a-453c-a560-982f6642fb5b" containerName="manager" containerID="cri-o://aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76" gracePeriod=30
Oct 30 21:59:38.196126 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.196097    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:38.322665 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.322618    4549 generic.go:356] "Generic (PLEG): container finished" podID="9402ffdf-7a1a-453c-a560-982f6642fb5b" containerID="aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76" exitCode=0
Oct 30 21:59:38.322890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.322734    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-bn92k" event={"ID":"9402ffdf-7a1a-453c-a560-982f6642fb5b","Type":"ContainerDied","Data":"aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76"}
Oct 30 21:59:38.322890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.322817    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-bn92k" event={"ID":"9402ffdf-7a1a-453c-a560-982f6642fb5b","Type":"ContainerDied","Data":"32b23b99baf9df089ce17146b86c6f3b02623c3f301982123bd34493ba8ae982"}
Oct 30 21:59:38.322890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.322750    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-bn92k"
Oct 30 21:59:38.322890 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.322871    4549 scope.go:122] "RemoveContainer" containerID="aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76"
Oct 30 21:59:38.332065 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.332039    4549 scope.go:122] "RemoveContainer" containerID="aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76"
Oct 30 21:59:38.332361 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:59:38.332343    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76\": container with ID starting with aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76 not found: ID does not exist" containerID="aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76"
Oct 30 21:59:38.332435 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.332371    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76"} err="failed to get container status \"aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76\": rpc error: code = NotFound desc = could not find container \"aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76\": container with ID starting with aad7b4ab275e167fbf82183676c5783319127586f8041b1c670abd2e154b3e76 not found: ID does not exist"
Oct 30 21:59:38.389669 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.389582    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b\" (UniqueName: \"kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b\") pod \"9402ffdf-7a1a-453c-a560-982f6642fb5b\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") "
Oct 30 21:59:38.389908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.389686    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs\") pod \"9402ffdf-7a1a-453c-a560-982f6642fb5b\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") "
Oct 30 21:59:38.389908 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.389805    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert\") pod \"9402ffdf-7a1a-453c-a560-982f6642fb5b\" (UID: \"9402ffdf-7a1a-453c-a560-982f6642fb5b\") "
Oct 30 21:59:38.392046 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.392002    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b" pod "9402ffdf-7a1a-453c-a560-982f6642fb5b" (UID: "9402ffdf-7a1a-453c-a560-982f6642fb5b"). InnerVolumeSpecName "kube-api-access-dwx4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:38.392177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.392044    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs" pod "9402ffdf-7a1a-453c-a560-982f6642fb5b" (UID: "9402ffdf-7a1a-453c-a560-982f6642fb5b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:38.392177 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.392099    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert" pod "9402ffdf-7a1a-453c-a560-982f6642fb5b" (UID: "9402ffdf-7a1a-453c-a560-982f6642fb5b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:38.490810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.490770    4549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-webhook-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:59:38.490810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.490803    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwx4b\" (UniqueName: \"kubernetes.io/projected/9402ffdf-7a1a-453c-a560-982f6642fb5b-kube-api-access-dwx4b\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:59:38.490810 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.490813    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9402ffdf-7a1a-453c-a560-982f6642fb5b-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 21:59:38.667859 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.667766    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-bn92k"]
Oct 30 21:59:38.679811 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:38.679770    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-bn92k"]
Oct 30 21:59:39.341951 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.341916    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9402ffdf-7a1a-453c-a560-982f6642fb5b" path="/var/lib/kubelet/pods/9402ffdf-7a1a-453c-a560-982f6642fb5b/volumes"
Oct 30 21:59:39.733422 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.733322    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:39.869118 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869076    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-654c5dd44b-gw588"]
Oct 30 21:59:39.869626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869602    4549 cpu_manager.go:414] "containerMap: removing container" podUID="9402ffdf-7a1a-453c-a560-982f6642fb5b" containerName="manager"
Oct 30 21:59:39.869626 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869627    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9402ffdf-7a1a-453c-a560-982f6642fb5b" containerName="manager"
Oct 30 21:59:39.869806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869660    4549 cpu_manager.go:414] "containerMap: removing container" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy"
Oct 30 21:59:39.869806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869668    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy"
Oct 30 21:59:39.869806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869771    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="0080a530-5406-4ecb-8fcc-ad4fa5fb0793" containerName="istio-proxy"
Oct 30 21:59:39.869806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869786    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="9402ffdf-7a1a-453c-a560-982f6642fb5b" containerName="manager"
Oct 30 21:59:39.869806 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.869792    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="71f46951-9378-4dee-9f2e-42d72da5ee1f" containerName="istio-proxy"
Oct 30 21:59:39.874067 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.874047    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:39.889175 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.889150    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mhc8s\""
Oct 30 21:59:39.901192 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.901157    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:59:39.902644 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.902623    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfb0755-61bd-4864-bc81-d79fccb5e66c-webhook-cert\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:39.902719 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.902651    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfh4c\" (UniqueName: \"kubernetes.io/projected/7cfb0755-61bd-4864-bc81-d79fccb5e66c-kube-api-access-pfh4c\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:39.902775 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.902753    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cfb0755-61bd-4864-bc81-d79fccb5e66c-metrics-certs\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:39.902819 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.902785    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-gw588"]
Oct 30 21:59:39.989914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.989803    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:59:39.989914 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:39.989808    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:59:40.003226 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.003191    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfb0755-61bd-4864-bc81-d79fccb5e66c-webhook-cert\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.003294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.003230    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfh4c\" (UniqueName: \"kubernetes.io/projected/7cfb0755-61bd-4864-bc81-d79fccb5e66c-kube-api-access-pfh4c\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.003294 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.003283    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cfb0755-61bd-4864-bc81-d79fccb5e66c-metrics-certs\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.005980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.005950    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfb0755-61bd-4864-bc81-d79fccb5e66c-webhook-cert\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.005980 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.005978    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cfb0755-61bd-4864-bc81-d79fccb5e66c-metrics-certs\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.023513 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.023477    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfh4c\" (UniqueName: \"kubernetes.io/projected/7cfb0755-61bd-4864-bc81-d79fccb5e66c-kube-api-access-pfh4c\") pod \"maas-controller-654c5dd44b-gw588\" (UID: \"7cfb0755-61bd-4864-bc81-d79fccb5e66c\") " pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.187173 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.187129    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:40.482704 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:40.482676    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-654c5dd44b-gw588"]
Oct 30 21:59:40.485638 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:59:40.485606    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfb0755_61bd_4864_bc81_d79fccb5e66c.slice/crio-1e28f69b07c985d5734979c3ff62a1e14a3b4d889bffc0e28d24c75a0ff279be WatchSource:0}: Error finding container 1e28f69b07c985d5734979c3ff62a1e14a3b4d889bffc0e28d24c75a0ff279be: Status 404 returned error can't find the container with id 1e28f69b07c985d5734979c3ff62a1e14a3b4d889bffc0e28d24c75a0ff279be
Oct 30 21:59:41.345167 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.345124    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-gw588" event={"ID":"7cfb0755-61bd-4864-bc81-d79fccb5e66c","Type":"ContainerStarted","Data":"eab61bf280a729084640472f4b95a30e562c6d536baa9775697e4b79d1ad8b26"}
Oct 30 21:59:41.345390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.345188    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-654c5dd44b-gw588" event={"ID":"7cfb0755-61bd-4864-bc81-d79fccb5e66c","Type":"ContainerStarted","Data":"1e28f69b07c985d5734979c3ff62a1e14a3b4d889bffc0e28d24c75a0ff279be"}
Oct 30 21:59:41.345390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.345250    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:41.385371 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.385308    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-654c5dd44b-gw588" podStartSLOduration=1.965678897 podStartE2EDuration="2.385288904s" totalImagesPullingTime="419.610007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:41.384257859 +0000 UTC m=+3564.158657799" watchObservedRunningTime="2026-10-30 21:59:41.385288904 +0000 UTC m=+3564.159688844"
Oct 30 21:59:41.821574 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.821535    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"]
Oct 30 21:59:41.825741 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.825720    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.840624 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.840604    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-w59m6\""
Oct 30 21:59:41.854712 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.854683    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"]
Oct 30 21:59:41.920705 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.920670    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.920950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.920739    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.920950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.920804    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plzpv\" (UniqueName: \"kubernetes.io/projected/2822d220-08fd-45d5-be5c-9ffa403c0f7e-kube-api-access-plzpv\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.920950 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.920870    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.921133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.920981    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.921133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.921019    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.921133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.921101    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.921133 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.921125    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:41.921293 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:41.921144    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022589 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022551    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022598    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plzpv\" (UniqueName: \"kubernetes.io/projected/2822d220-08fd-45d5-be5c-9ffa403c0f7e-kube-api-access-plzpv\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022630    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022778 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022660    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022876    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022944    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.022972 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022968    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.022995    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.023039    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.023063    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023160 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.023130    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023411 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.023343    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023616 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.023596    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.023706 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.023641    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.025358 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.025335    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.025595 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.025574    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.042880 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.042845    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plzpv\" (UniqueName: \"kubernetes.io/projected/2822d220-08fd-45d5-be5c-9ffa403c0f7e-kube-api-access-plzpv\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.043039 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.042856    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2822d220-08fd-45d5-be5c-9ffa403c0f7e-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-h2mcv\" (UID: \"2822d220-08fd-45d5-be5c-9ffa403c0f7e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.138729 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.138639    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:42.456896 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.456804    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"]
Oct 30 21:59:42.459089 ip-10-0-140-179 kubenswrapper[4549]: W1030 21:59:42.459061    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2822d220_08fd_45d5_be5c_9ffa403c0f7e.slice/crio-5fe52365799f53809a3a42210915975d657f9dfee4444d7616ebdf2a6faa3525 WatchSource:0}: Error finding container 5fe52365799f53809a3a42210915975d657f9dfee4444d7616ebdf2a6faa3525: Status 404 returned error can't find the container with id 5fe52365799f53809a3a42210915975d657f9dfee4444d7616ebdf2a6faa3525
Oct 30 21:59:42.461140 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.461109    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:42.461217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.461170    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:42.461217 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:42.461198    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:43.355260 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:43.355222    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" event={"ID":"2822d220-08fd-45d5-be5c-9ffa403c0f7e","Type":"ContainerStarted","Data":"ee9ab4bc01684e8943791070a3c189a6f6ee9a33c755481cd054b734fdda8164"}
Oct 30 21:59:43.355703 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:43.355279    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" event={"ID":"2822d220-08fd-45d5-be5c-9ffa403c0f7e","Type":"ContainerStarted","Data":"5fe52365799f53809a3a42210915975d657f9dfee4444d7616ebdf2a6faa3525"}
Oct 30 21:59:43.428034 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:43.427978    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podStartSLOduration=2.427964212 podStartE2EDuration="2.427964212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:43.427500427 +0000 UTC m=+3566.201900358" watchObservedRunningTime="2026-10-30 21:59:43.427964212 +0000 UTC m=+3566.202364148"
Oct 30 21:59:44.139444 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:44.139403    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:44.141098 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:44.141070    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused" start-of-body=
Oct 30 21:59:44.141230 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:44.141125    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podUID="2822d220-08fd-45d5-be5c-9ffa403c0f7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused"
Oct 30 21:59:45.140033 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:45.139994    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused" start-of-body=
Oct 30 21:59:45.140496 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:45.140051    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podUID="2822d220-08fd-45d5-be5c-9ffa403c0f7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused"
Oct 30 21:59:46.139567 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:46.139521    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused" start-of-body=
Oct 30 21:59:46.139765 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:46.139586    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podUID="2822d220-08fd-45d5-be5c-9ffa403c0f7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused"
Oct 30 21:59:47.139390 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:47.139353    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused" start-of-body=
Oct 30 21:59:47.139851 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:47.139426    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podUID="2822d220-08fd-45d5-be5c-9ffa403c0f7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused"
Oct 30 21:59:48.139952 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:48.139911    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused" start-of-body=
Oct 30 21:59:48.140393 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:48.139998    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podUID="2822d220-08fd-45d5-be5c-9ffa403c0f7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused"
Oct 30 21:59:49.139663 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:49.139625    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused" start-of-body=
Oct 30 21:59:49.139861 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:49.139696    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv" podUID="2822d220-08fd-45d5-be5c-9ffa403c0f7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.231:15021/healthz/ready\": dial tcp 10.133.0.231:15021: connect: connection refused"
Oct 30 21:59:50.653340 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:50.653303    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:50.653879 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:50.653717    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:50.654631 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:50.654610    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv"
Oct 30 21:59:52.351059 ip-10-0-140-179 kubenswrapper[4549]: I1030 21:59:52.351027    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-654c5dd44b-gw588"
Oct 30 21:59:56.574995 ip-10-0-140-179 kubenswrapper[4549]: E1030 21:59:56.574913    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:00:11.781345 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:11.781282    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:18.892106 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:18.892070    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:00:18.904470 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:18.904438    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:00:26.573870 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:00:26.573801    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:00:29.970277 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:29.970242    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:42.006440 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.006407    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"]
Oct 30 22:00:42.009797 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.009782    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.039181 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.039144    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"]
Oct 30 22:00:42.115739 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.115702    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d5e344-c230-4e7e-a922-e7868c0038d4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.115938 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.115748    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.115938 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.115899    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.116051 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.115938    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.116051 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.115960    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.116051 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.115998    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxzg\" (UniqueName: \"kubernetes.io/projected/20d5e344-c230-4e7e-a922-e7868c0038d4-kube-api-access-2wxzg\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.116160 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.116070    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.116160 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.116153    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.116238 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.116174    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217298 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217251    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxzg\" (UniqueName: \"kubernetes.io/projected/20d5e344-c230-4e7e-a922-e7868c0038d4-kube-api-access-2wxzg\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217310    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217350    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217368    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217396    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d5e344-c230-4e7e-a922-e7868c0038d4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217423    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217445    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217886 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217747    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217905    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217918    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.217963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.217857    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.218140 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.218081    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.218240 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.218220    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.218331 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.218306    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20d5e344-c230-4e7e-a922-e7868c0038d4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.220257 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.220234    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.220368 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.220300    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.236435 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.236398    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20d5e344-c230-4e7e-a922-e7868c0038d4-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.236570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.236551    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxzg\" (UniqueName: \"kubernetes.io/projected/20d5e344-c230-4e7e-a922-e7868c0038d4-kube-api-access-2wxzg\") pod \"maas-default-gateway-openshift-default-57f78bcd55-p66fx\" (UID: \"20d5e344-c230-4e7e-a922-e7868c0038d4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.324480 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.324448    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:42.637149 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.637118    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"]
Oct 30 22:00:42.643748 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.643703    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:00:42.644054 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.644023    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:00:42.644252 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.644231    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:00:42.649721 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:42.649684    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" event={"ID":"20d5e344-c230-4e7e-a922-e7868c0038d4","Type":"ContainerStarted","Data":"a29db14c1202d25e46a58f4655ea8c6527baf126f6a046a38ae7adf35643ad4f"}
Oct 30 22:00:43.655071 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:43.655031    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" event={"ID":"20d5e344-c230-4e7e-a922-e7868c0038d4","Type":"ContainerStarted","Data":"3cd49d73b50e847132ad2e86734446a668f9129741cfec84ef008c96c3c0ed69"}
Oct 30 22:00:43.729378 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:43.729315    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podStartSLOduration=2.729301168 podStartE2EDuration="2.729301168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:43.716738094 +0000 UTC m=+3626.491138034" watchObservedRunningTime="2026-10-30 22:00:43.729301168 +0000 UTC m=+3626.503701107"
Oct 30 22:00:44.325392 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:44.325348    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:44.327079 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:44.327049    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused" start-of-body=
Oct 30 22:00:44.327231 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:44.327106    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podUID="20d5e344-c230-4e7e-a922-e7868c0038d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused"
Oct 30 22:00:45.324899 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:45.324866    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused" start-of-body=
Oct 30 22:00:45.325322 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:45.324918    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podUID="20d5e344-c230-4e7e-a922-e7868c0038d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused"
Oct 30 22:00:46.325237 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:46.325202    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused" start-of-body=
Oct 30 22:00:46.325759 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:46.325255    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podUID="20d5e344-c230-4e7e-a922-e7868c0038d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused"
Oct 30 22:00:47.325541 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:47.325499    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused" start-of-body=
Oct 30 22:00:47.326093 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:47.325554    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podUID="20d5e344-c230-4e7e-a922-e7868c0038d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused"
Oct 30 22:00:48.325541 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:48.325505    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused" start-of-body=
Oct 30 22:00:48.325741 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:48.325565    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podUID="20d5e344-c230-4e7e-a922-e7868c0038d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused"
Oct 30 22:00:48.465389 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:48.465335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:48.598528 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:48.598440    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:49.325593 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:49.325560    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused" start-of-body=
Oct 30 22:00:49.325825 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:49.325624    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx" podUID="20d5e344-c230-4e7e-a922-e7868c0038d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.232:15021/healthz/ready\": dial tcp 10.133.0.232:15021: connect: connection refused"
Oct 30 22:00:50.840948 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:50.840914    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:50.841419 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:50.841368    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:50.842273 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:00:50.842250    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx"
Oct 30 22:00:56.574873 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:00:56.574806    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:01:11.928644 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:11.928592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:12.094761 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.094712    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"]
Oct 30 22:01:12.102155 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.102124    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.130852 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.121896    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"]
Oct 30 22:01:12.244581 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244477    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.244581 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244520    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.244581 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244552    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.244953 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244617    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.244953 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244758    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.244953 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244914    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.244953 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.244945    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.245150 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.245021    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdgqw\" (UniqueName: \"kubernetes.io/projected/98b392c7-af0b-4e20-bf43-dbd024ba92b4-kube-api-access-qdgqw\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.245150 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.245093    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.345728 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345679    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.345728 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345727    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345747    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345775    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345799    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345841    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345915    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345933    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.345962    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdgqw\" (UniqueName: \"kubernetes.io/projected/98b392c7-af0b-4e20-bf43-dbd024ba92b4-kube-api-access-qdgqw\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346454 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.346306    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346454 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.346433    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346634 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.346573    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.346708 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.346622    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.347317 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.347293    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.348413 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.348386    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.348736 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.348714    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.367611 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.367576    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98b392c7-af0b-4e20-bf43-dbd024ba92b4-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.367763 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.367585    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdgqw\" (UniqueName: \"kubernetes.io/projected/98b392c7-af0b-4e20-bf43-dbd024ba92b4-kube-api-access-qdgqw\") pod \"maas-default-gateway-openshift-default-57f78bcd55-pnzz7\" (UID: \"98b392c7-af0b-4e20-bf43-dbd024ba92b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.434039 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.433968    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:12.769999 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.769975    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"]
Oct 30 22:01:12.772409 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:01:12.772383    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b392c7_af0b_4e20_bf43_dbd024ba92b4.slice/crio-29d173f4d6e6894f62525e818daa3b1875eac5aa7e58e2daeef382c16e2669c9 WatchSource:0}: Error finding container 29d173f4d6e6894f62525e818daa3b1875eac5aa7e58e2daeef382c16e2669c9: Status 404 returned error can't find the container with id 29d173f4d6e6894f62525e818daa3b1875eac5aa7e58e2daeef382c16e2669c9
Oct 30 22:01:12.774574 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.774538    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:12.774690 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.774623    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:12.774690 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.774669    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:12.800172 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:12.800138    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" event={"ID":"98b392c7-af0b-4e20-bf43-dbd024ba92b4","Type":"ContainerStarted","Data":"29d173f4d6e6894f62525e818daa3b1875eac5aa7e58e2daeef382c16e2669c9"}
Oct 30 22:01:13.806134 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:13.806094    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" event={"ID":"98b392c7-af0b-4e20-bf43-dbd024ba92b4","Type":"ContainerStarted","Data":"805ea965a17ab2246728bb077f818209a9c2fee94102cb1ccf853fd9c8e7a089"}
Oct 30 22:01:13.861518 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:13.861455    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podStartSLOduration=1.86143976 podStartE2EDuration="1.86143976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:13.861049313 +0000 UTC m=+3656.635449254" watchObservedRunningTime="2026-10-30 22:01:13.86143976 +0000 UTC m=+3656.635839699"
Oct 30 22:01:14.434104 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:14.434065    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:14.435691 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:14.435665    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused" start-of-body=
Oct 30 22:01:14.435859 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:14.435723    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused"
Oct 30 22:01:15.434987 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:15.434952    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused" start-of-body=
Oct 30 22:01:15.435404 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:15.435017    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused"
Oct 30 22:01:16.434399 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:16.434358    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused" start-of-body=
Oct 30 22:01:16.434618 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:16.434416    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused"
Oct 30 22:01:17.434450 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:17.434408    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused" start-of-body=
Oct 30 22:01:17.434865 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:17.434475    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused"
Oct 30 22:01:18.434555 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:18.434513    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused" start-of-body=
Oct 30 22:01:18.435028 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:18.434576    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused"
Oct 30 22:01:19.434596 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:19.434562    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused" start-of-body=
Oct 30 22:01:19.435205 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:19.434636    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": dial tcp 10.133.0.233:15021: connect: connection refused"
Oct 30 22:01:21.130033 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:21.129982    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:21.130574 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:21.130258    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:21.131170 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:21.131147    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7"
Oct 30 22:01:26.573449 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:01:26.573409    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:01:27.120744 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.120709    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"]
Oct 30 22:01:27.125239 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.125214    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.153875 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.153811    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"]
Oct 30 22:01:27.217570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217534    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217760 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217578    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217760 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217688    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217760 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217716    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217760 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217741    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbhcc\" (UniqueName: \"kubernetes.io/projected/579c215d-3fff-4eba-8e6b-52b2054a8d0c-kube-api-access-mbhcc\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217938 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217788    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217938 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217875    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217938 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217902    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.217938 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.217923    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318340 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318306    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318340 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318344    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318368    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbhcc\" (UniqueName: \"kubernetes.io/projected/579c215d-3fff-4eba-8e6b-52b2054a8d0c-kube-api-access-mbhcc\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318390    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318415    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318438    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318456    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318485    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.318698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318518    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.319124 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318916    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.319124 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318936    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.319124 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.318983    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.319124 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.319040    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.319443 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.319423    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.321100 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.321069    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.321212 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.321184    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.337352 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.337316    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579c215d-3fff-4eba-8e6b-52b2054a8d0c-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.337620 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.337600    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbhcc\" (UniqueName: \"kubernetes.io/projected/579c215d-3fff-4eba-8e6b-52b2054a8d0c-kube-api-access-mbhcc\") pod \"maas-default-gateway-openshift-default-57f78bcd55-nb8vx\" (UID: \"579c215d-3fff-4eba-8e6b-52b2054a8d0c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.438898 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.438785    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:27.747741 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.747531    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"]
Oct 30 22:01:27.750605 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:01:27.750578    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579c215d_3fff_4eba_8e6b_52b2054a8d0c.slice/crio-6b18c07dbe501b3fd79ad051e75cca20bea6d31606aa213147fd49c05e349049 WatchSource:0}: Error finding container 6b18c07dbe501b3fd79ad051e75cca20bea6d31606aa213147fd49c05e349049: Status 404 returned error can't find the container with id 6b18c07dbe501b3fd79ad051e75cca20bea6d31606aa213147fd49c05e349049
Oct 30 22:01:27.752975 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.752936    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:27.753080 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.753013    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:27.753080 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.753043    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:27.869455 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.869414    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" event={"ID":"579c215d-3fff-4eba-8e6b-52b2054a8d0c","Type":"ContainerStarted","Data":"64e8d8ef9068bb833500a024bb8a9b6d39117746d46f194985438329f4cf6b70"}
Oct 30 22:01:27.869663 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.869482    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" event={"ID":"579c215d-3fff-4eba-8e6b-52b2054a8d0c","Type":"ContainerStarted","Data":"6b18c07dbe501b3fd79ad051e75cca20bea6d31606aa213147fd49c05e349049"}
Oct 30 22:01:27.911538 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:27.911477    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podStartSLOduration=0.911461015 podStartE2EDuration="911.461015ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:27.91045708 +0000 UTC m=+3670.684857020" watchObservedRunningTime="2026-10-30 22:01:27.911461015 +0000 UTC m=+3670.685861022"
Oct 30 22:01:28.439716 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:28.439675    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:28.441315 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:28.441288    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:28.441444 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:28.441344    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:29.011905 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:29.011817    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:29.439344 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:29.439302    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:29.439544 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:29.439379    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:30.439249 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:30.439217    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:30.439687 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:30.439273    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:31.440068 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:31.440024    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:31.440508 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:31.440091    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:32.439738 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:32.439696    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:32.439991 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:32.439760    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:33.439660 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:33.439628    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:33.440164 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:33.439687    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:34.439518 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:34.439474    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused" start-of-body=
Oct 30 22:01:34.439754 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:34.439536    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": dial tcp 10.133.0.234:15021: connect: connection refused"
Oct 30 22:01:36.490852 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:36.490005    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:01:36.490852 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:36.490111    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:37.497857 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:37.496939    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:01:37.497857 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:37.497010    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:38.497921 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:38.497880    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:01:38.498499 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:38.497963    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:01:39.091020 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:39.090988    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:39.091523 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:39.091504    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:39.092318 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:01:39.092301    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx"
Oct 30 22:01:56.572958 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:01:56.572918    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:02:12.087854 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:02:12.087655    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:26.573386 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:02:26.573346    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:02:31.538662 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:02:31.538559    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:44.358430 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:02:44.358391    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:44.481730 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:02:44.481687    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:56.573104 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:02:56.573058    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:03:12.195882 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:03:12.195815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:22.178602 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:03:22.178558    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.233:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:03:22.179477 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:03:22.179432    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7" podUID="98b392c7-af0b-4e20-bf43-dbd024ba92b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.233:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:26.572801 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:03:26.572757    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:03:55.377900 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:03:55.377866    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:55.566918 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:03:55.566884    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:56.576350 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:03:56.576310    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:04:12.424856 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:12.424790    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:20.053689 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:20.053625    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:26.573059 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:04:26.573019    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:04:40.033056 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.033020    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:40.461228 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.461164    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"]
Oct 30 22:04:40.464785 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.464760    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.481138 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.481110    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-d36cd3-openshift-default-dockercfg-g628v\""
Oct 30 22:04:40.496502 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.496460    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"]
Oct 30 22:04:40.571317 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.571273    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.571663 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.571644    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.571858 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.571819    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.572081 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.572064    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.572308 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.572283    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hdxb\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.572442 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.572326    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.572442 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.572372    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.572442 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.572434    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.572574 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.572482    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.589803 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.589768    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:40.673205 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673171    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673417 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673219    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673417 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673248    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673417 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673281    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673417 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673305    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673417 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673323    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673417 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673383    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hdxb\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673764 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673559    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673764 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673606    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673764 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673728    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673976 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673798    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673976 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673879    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.673976 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.673946    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.674109 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.674044    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.676109 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.676092    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.676241 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.676190    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.695037 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.694996    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.695153 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.695059    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hdxb\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb\") pod \"e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") " pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.780665 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.780551    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:40.780843 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:40.780739    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:41.096933 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:41.096904    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"]
Oct 30 22:04:41.098740 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:04:41.098702    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450bb602_3745_44b7_874e_4a64804eca48.slice/crio-4c07af2c81aef205ae8f53c4b900a0449fb9c5fe4bb228ad5d4300e072923122 WatchSource:0}: Error finding container 4c07af2c81aef205ae8f53c4b900a0449fb9c5fe4bb228ad5d4300e072923122: Status 404 returned error can't find the container with id 4c07af2c81aef205ae8f53c4b900a0449fb9c5fe4bb228ad5d4300e072923122
Oct 30 22:04:41.101295 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:41.101255    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:41.101383 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:41.101344    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:41.101424 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:41.101387    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.108226 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:42.108187    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" event={"ID":"450bb602-3745-44b7-874e-4a64804eca48","Type":"ContainerStarted","Data":"7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"}
Oct 30 22:04:42.108659 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:42.108274    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" event={"ID":"450bb602-3745-44b7-874e-4a64804eca48","Type":"ContainerStarted","Data":"4c07af2c81aef205ae8f53c4b900a0449fb9c5fe4bb228ad5d4300e072923122"}
Oct 30 22:04:42.167143 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:42.167082    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podStartSLOduration=2.16706883 podStartE2EDuration="2.16706883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:42.16650065 +0000 UTC m=+3864.940900589" watchObservedRunningTime="2026-10-30 22:04:42.16706883 +0000 UTC m=+3864.941468770"
Oct 30 22:04:42.781735 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:42.781698    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:42.786987 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:42.786963    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:43.113257 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:43.113225    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:43.114469 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:43.114447    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:04:44.686630 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:44.686594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:44.968670 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:44.968586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:46.123780 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.123731    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:46.273031 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.272990    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:46.446126 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.446035    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:46.543435 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.543400    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"]
Oct 30 22:04:46.552227 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.552196    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.576497 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.576463    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"]
Oct 30 22:04:46.585737 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.585703    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-459ec8f2-dockercfg-gwtk2\""
Oct 30 22:04:46.586001 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.585849    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-d36cd3\""
Oct 30 22:04:46.739061 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.738951    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-778lb\" (UniqueName: \"kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb\") pod \"payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.739253 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.739071    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.840140 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.840106    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-778lb\" (UniqueName: \"kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb\") pod \"payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.840358 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.840155    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.841260 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.841235    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.860047 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.860016    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-778lb\" (UniqueName: \"kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb\") pod \"payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:46.872901 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:46.872868    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:47.252394 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:47.252357    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"]
Oct 30 22:04:47.253547 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:04:47.253515    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea24fd63_39c9_4880_971c_ccd3207d7e3e.slice/crio-372d2ac9255d425bf412f612a1c42b5dfff20f27e41c567cd38d5875358e88aa WatchSource:0}: Error finding container 372d2ac9255d425bf412f612a1c42b5dfff20f27e41c567cd38d5875358e88aa: Status 404 returned error can't find the container with id 372d2ac9255d425bf412f612a1c42b5dfff20f27e41c567cd38d5875358e88aa
Oct 30 22:04:47.983598 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:47.983559    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:48.139006 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:48.138966    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" event={"ID":"ea24fd63-39c9-4880-971c-ccd3207d7e3e","Type":"ContainerStarted","Data":"12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa"}
Oct 30 22:04:48.139206 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:48.139030    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" event={"ID":"ea24fd63-39c9-4880-971c-ccd3207d7e3e","Type":"ContainerStarted","Data":"372d2ac9255d425bf412f612a1c42b5dfff20f27e41c567cd38d5875358e88aa"}
Oct 30 22:04:48.139206 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:48.139066    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:48.185027 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:48.184976    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" podStartSLOduration=2.184961579 podStartE2EDuration="2.184961579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:48.183530376 +0000 UTC m=+3870.957930312" watchObservedRunningTime="2026-10-30 22:04:48.184961579 +0000 UTC m=+3870.959361517"
Oct 30 22:04:49.658819 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:49.658755    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:51.309907 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:51.298632    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:52.849629 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:52.849592    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:54.531134 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:54.531091    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:56.262708 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:56.262670    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:56.574473 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:04:56.574441    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:04:57.745697 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:57.745653    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:59.147029 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:59.146994    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:04:59.154995 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:59.154961    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.235:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:59.155161 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:59.155020    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.235:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:59.371331 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:59.371279    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:59.586741 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:04:59.586686    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:00.218240 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.218206    4549 generic.go:356] "Generic (PLEG): container finished" podID="450bb602-3745-44b7-874e-4a64804eca48" containerID="7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4" exitCode=0
Oct 30 22:05:00.218664 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.218274    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" event={"ID":"450bb602-3745-44b7-874e-4a64804eca48","Type":"ContainerDied","Data":"7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"}
Oct 30 22:05:00.218735 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.218674    4549 scope.go:122] "RemoveContainer" containerID="7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"
Oct 30 22:05:00.219974 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.219943    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:00.220073 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.220011    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:00.220073 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.220046    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:00.242475 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.242444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:00.586733 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:00.586688    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:01.224119 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:01.224078    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" event={"ID":"450bb602-3745-44b7-874e-4a64804eca48","Type":"ContainerStarted","Data":"61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850"}
Oct 30 22:05:01.395242 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:01.395209    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:01.786758 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:01.786267    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:05:01.786758 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:01.786406    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused" start-of-body=
Oct 30 22:05:01.786758 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:01.786456    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused"
Oct 30 22:05:02.791330 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:02.791288    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused" start-of-body=
Oct 30 22:05:02.792067 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:02.792030    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused"
Oct 30 22:05:02.845921 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:02.845890    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:03.333409 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:03.333376    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:03.784375 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:03.784275    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused" start-of-body=
Oct 30 22:05:03.784681 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:03.784647    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused"
Oct 30 22:05:04.788857 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:04.783318    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused" start-of-body=
Oct 30 22:05:04.788857 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:04.783385    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused"
Oct 30 22:05:05.094855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:05.093952    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:05.784857 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:05.784295    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused" start-of-body=
Oct 30 22:05:05.784857 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:05.784365    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.235:15021/healthz/ready\": dial tcp 10.133.0.235:15021: connect: connection refused"
Oct 30 22:05:06.735812 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:06.735775    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:06.800884 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:06.800814    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:05:06.801359 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:06.801331    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:05:06.803351 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:06.803323    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:05:08.443084 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:08.443052    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:09.833076 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:09.833036    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:11.454134 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:11.454084    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:12.452122 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:12.452081    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:12.927633 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:12.927591    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:14.434196 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:14.434155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:15.930807 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:15.930774    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:17.472630 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:17.472578    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:18.907175 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:18.907139    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:18.922991 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:18.922964    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:05:18.960645 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:18.960612    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:05:20.463651 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:20.463615    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:22.327476 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:22.327430    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:23.984653 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:23.984611    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:25.645382 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:25.645344    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:26.573772 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:05:26.573735    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:05:27.151549 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:27.151516    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:28.769632 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:28.769299    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:30.364609 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:30.364548    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:31.924364 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:31.924329    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:33.504097 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:33.504060    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:34.902370 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:34.902335    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:36.303921 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:36.303879    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:37.709014 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:37.708963    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:39.118694 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:39.118662    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:40.526877 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:40.526809    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:42.024351 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:42.024304    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:43.527532 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:43.527493    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:45.033349 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:45.033262    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:45.159587 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:45.159542    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:46.683532 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:46.683496    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:48.278154 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:48.278104    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:49.719488 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:49.719448    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:50.306201 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:50.306157    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:51.360540 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:51.360443    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:52.886995 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:52.886962    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:54.231054 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:54.231020    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:55.720233 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:55.720197    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:55.763959 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:55.763921    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"]
Oct 30 22:05:55.929108 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:55.929078    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:56.049601 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:56.049513    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:56.573959 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:05:56.573917    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:05:57.103200 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:57.103131    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:58.317180 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:58.317146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:58.734501 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:58.734401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:58.857332 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:58.857244    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:59.352633 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:05:59.352602    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:00.768168 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:00.768127    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerName="payload-processing" containerID="cri-o://12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa" gracePeriod=25
Oct 30 22:06:05.347823 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:05.347790    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.497951 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:05.497915    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.632491 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:05.632402    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"]
Oct 30 22:06:05.632819 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:05.632705    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" containerID="cri-o://61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850" gracePeriod=30
Oct 30 22:06:05.697753 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:05.697711    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:06.803974 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:06.803933    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:06:06.804382 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:06.803998    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:06:09.146011 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:09.145967    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:06:10.893158 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:10.893132    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:06:11.055295 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055204    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055295 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055286    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055343    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055373    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055417    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055541    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:11.055570 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055555    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055879 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055623    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:11.055879 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055659    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055879 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055737    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055879 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055769    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket\") pod \"450bb602-3745-44b7-874e-4a64804eca48\" (UID: \"450bb602-3745-44b7-874e-4a64804eca48\") "
Oct 30 22:06:11.055879 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055856    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:11.055879 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.055873    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:06:11.056235 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.056154    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.056235 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.056177    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.056235 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.056190    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450bb602-3745-44b7-874e-4a64804eca48-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.056235 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.056195    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:11.056235 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.056204    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.058066 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.058042    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:11.058195 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.058128    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:06:11.058559 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.058541    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "kube-api-access-4hdxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:11.058610 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.058568    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token" pod "450bb602-3745-44b7-874e-4a64804eca48" (UID: "450bb602-3745-44b7-874e-4a64804eca48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:11.157014 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.156973    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.157014 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.157006    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.157014 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.157015    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450bb602-3745-44b7-874e-4a64804eca48-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.157014 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.157025    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hdxb\" (UniqueName: \"kubernetes.io/projected/450bb602-3745-44b7-874e-4a64804eca48-kube-api-access-4hdxb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.157323 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.157034    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450bb602-3745-44b7-874e-4a64804eca48-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:11.570564 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.570527    4549 generic.go:356] "Generic (PLEG): container finished" podID="450bb602-3745-44b7-874e-4a64804eca48" containerID="61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850" exitCode=0
Oct 30 22:06:11.570770 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.570583    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"
Oct 30 22:06:11.570770 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.570589    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" event={"ID":"450bb602-3745-44b7-874e-4a64804eca48","Type":"ContainerDied","Data":"61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850"}
Oct 30 22:06:11.570770 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.570635    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf" event={"ID":"450bb602-3745-44b7-874e-4a64804eca48","Type":"ContainerDied","Data":"4c07af2c81aef205ae8f53c4b900a0449fb9c5fe4bb228ad5d4300e072923122"}
Oct 30 22:06:11.570770 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.570657    4549 scope.go:122] "RemoveContainer" containerID="61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850"
Oct 30 22:06:11.581858 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.581813    4549 scope.go:122] "RemoveContainer" containerID="7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"
Oct 30 22:06:11.590745 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.590723    4549 scope.go:122] "RemoveContainer" containerID="61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850"
Oct 30 22:06:11.591140 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:06:11.591120    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850\": container with ID starting with 61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850 not found: ID does not exist" containerID="61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850"
Oct 30 22:06:11.591205 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.591150    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850"} err="failed to get container status \"61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850\": rpc error: code = NotFound desc = could not find container \"61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850\": container with ID starting with 61acd980071a61a84f01cd86974945735e4e7b7b6f3ba63cc49e930ac84b1850 not found: ID does not exist"
Oct 30 22:06:11.591205 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.591168    4549 scope.go:122] "RemoveContainer" containerID="7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"
Oct 30 22:06:11.591444 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:06:11.591424    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4\": container with ID starting with 7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4 not found: ID does not exist" containerID="7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"
Oct 30 22:06:11.591480 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.591452    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4"} err="failed to get container status \"7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4\": rpc error: code = NotFound desc = could not find container \"7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4\": container with ID starting with 7c3dfe897a3322d4b0440361aa78cd0a6e76d40b28772f5bcd1ebf090cfc48e4 not found: ID does not exist"
Oct 30 22:06:11.634015 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.633936    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"]
Oct 30 22:06:11.646190 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:11.646159    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf"]
Oct 30 22:06:12.557771 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:12.557705    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:13.350221 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:13.350175    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="450bb602-3745-44b7-874e-4a64804eca48" path="/var/lib/kubelet/pods/450bb602-3745-44b7-874e-4a64804eca48/volumes"
Oct 30 22:06:19.145170 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:19.145127    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:06:21.025365 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.025333    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:06:21.054968 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.054931    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume\") pod \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") "
Oct 30 22:06:21.055163 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.055025    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb\" (UniqueName: \"kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb\") pod \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\" (UID: \"ea24fd63-39c9-4880-971c-ccd3207d7e3e\") "
Oct 30 22:06:21.055329 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.055304    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume" pod "ea24fd63-39c9-4880-971c-ccd3207d7e3e" (UID: "ea24fd63-39c9-4880-971c-ccd3207d7e3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:06:21.057297 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.057272    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb" pod "ea24fd63-39c9-4880-971c-ccd3207d7e3e" (UID: "ea24fd63-39c9-4880-971c-ccd3207d7e3e"). InnerVolumeSpecName "kube-api-access-778lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:21.155822 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.155784    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea24fd63-39c9-4880-971c-ccd3207d7e3e-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:21.155822 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.155815    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-778lb\" (UniqueName: \"kubernetes.io/projected/ea24fd63-39c9-4880-971c-ccd3207d7e3e-kube-api-access-778lb\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:06:21.617305 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.617214    4549 generic.go:356] "Generic (PLEG): container finished" podID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerID="12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa" exitCode=0
Oct 30 22:06:21.617305 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.617257    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"
Oct 30 22:06:21.617507 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.617302    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" event={"ID":"ea24fd63-39c9-4880-971c-ccd3207d7e3e","Type":"ContainerDied","Data":"12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa"}
Oct 30 22:06:21.617507 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.617350    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54" event={"ID":"ea24fd63-39c9-4880-971c-ccd3207d7e3e","Type":"ContainerDied","Data":"372d2ac9255d425bf412f612a1c42b5dfff20f27e41c567cd38d5875358e88aa"}
Oct 30 22:06:21.617507 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.617370    4549 scope.go:122] "RemoveContainer" containerID="12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa"
Oct 30 22:06:21.626029 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.626012    4549 scope.go:122] "RemoveContainer" containerID="12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa"
Oct 30 22:06:21.626319 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:06:21.626299    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa\": container with ID starting with 12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa not found: ID does not exist" containerID="12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa"
Oct 30 22:06:21.626390 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.626334    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa"} err="failed to get container status \"12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa\": rpc error: code = NotFound desc = could not find container \"12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa\": container with ID starting with 12f5aec3f624f858ef7aa5d583d78755958d6c245d551f6eb273f133002cf5aa not found: ID does not exist"
Oct 30 22:06:21.668815 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.668775    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"]
Oct 30 22:06:21.682034 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:21.681981    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54"]
Oct 30 22:06:23.342531 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:23.342499    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" path="/var/lib/kubelet/pods/ea24fd63-39c9-4880-971c-ccd3207d7e3e/volumes"
Oct 30 22:06:26.573109 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:06:26.573069    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:06:27.327621 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:27.327582    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:28.396593 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:28.396557    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:33.661412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:33.661333    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:52.004227 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:06:52.004185    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:56.573973 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:06:56.573934    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:07:12.662634 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:07:12.662594    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:26.574951 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:07:26.574909    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:07:56.572565 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:07:56.572526    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:08:06.742186 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:06.742146    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:12.787500 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:12.787471    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:22.117400 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:22.117363    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:25.226002 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.225969    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:25.620895 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.620854    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"]
Oct 30 22:08:25.621573 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.621548    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerName="payload-processing"
Oct 30 22:08:25.621624 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.621579    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerName="payload-processing"
Oct 30 22:08:25.621624 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.621608    4549 cpu_manager.go:414] "containerMap: removing container" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy"
Oct 30 22:08:25.621624 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.621618    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy"
Oct 30 22:08:25.621802 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.621783    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy"
Oct 30 22:08:25.621861 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.621818    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea24fd63-39c9-4880-971c-ccd3207d7e3e" containerName="payload-processing"
Oct 30 22:08:25.625639 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.625615    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.641009 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.640980    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-9b97ce-openshift-default-dockercfg-lg8qq\""
Oct 30 22:08:25.655315 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.655281    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"]
Oct 30 22:08:25.732184 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.732142    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:25.790837 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.790788    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791049 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.790853    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791049 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.790872    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791049 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.791009    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791194 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.791048    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791194 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.791105    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkcm5\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791194 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.791176    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791318 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.791198    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.791318 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.791216    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.870524 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.870488    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:25.892788 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892714    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.892788 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892750    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.892788 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892769    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893070 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892886    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893070 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892921    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893070 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892939    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893070 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.892979    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893070 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893010    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893070 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893067    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkcm5\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893405 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893227    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893405 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893292    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893405 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893379    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893584 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893441    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.893653 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.893627    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.895623 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.895601    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.895850 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.895818    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.912592 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.912561    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.912700 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.912646    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkcm5\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5\") pod \"e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") " pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:25.941024 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:25.940993    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:26.255313 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:26.255280    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"]
Oct 30 22:08:26.256334 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:08:26.256309    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bbcc92_3e2a_46dd_bddc_4edf7c31629b.slice/crio-95d4a133b522bffd6fd89864970b5f643a6f12dd2ec6f3d7d81af846f28c2421 WatchSource:0}: Error finding container 95d4a133b522bffd6fd89864970b5f643a6f12dd2ec6f3d7d81af846f28c2421: Status 404 returned error can't find the container with id 95d4a133b522bffd6fd89864970b5f643a6f12dd2ec6f3d7d81af846f28c2421
Oct 30 22:08:26.258611 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:26.258577    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:26.258698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:26.258663    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:26.258698 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:26.258690    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:26.267319 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:26.267294    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" event={"ID":"05bbcc92-3e2a-46dd-bddc-4edf7c31629b","Type":"ContainerStarted","Data":"95d4a133b522bffd6fd89864970b5f643a6f12dd2ec6f3d7d81af846f28c2421"}
Oct 30 22:08:26.574956 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:08:26.574916    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:08:27.273913 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:27.273867    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" event={"ID":"05bbcc92-3e2a-46dd-bddc-4edf7c31629b","Type":"ContainerStarted","Data":"41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"}
Oct 30 22:08:27.317682 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:27.317629    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podStartSLOduration=2.317615266 podStartE2EDuration="2.317615266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:27.31555673 +0000 UTC m=+4090.089956672" watchObservedRunningTime="2026-10-30 22:08:27.317615266 +0000 UTC m=+4090.092015205"
Oct 30 22:08:27.941882 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:27.941814    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:27.946848 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:27.946801    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:28.279127 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:28.279035    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:28.279981 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:28.279954    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:29.742501 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:29.742466    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:30.014535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:30.014444    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:31.180275 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:31.180237    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:31.346855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:31.328857    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:31.480370 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:31.480281    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:32.823428 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:32.823372    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:34.323013 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:34.322959    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:35.830265 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:35.830227    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:37.421022 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:37.420974    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:38.887771 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:38.887694    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:40.388854 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:40.383299    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:41.940182 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:41.940143    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:43.435985 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:43.435950    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:44.320037 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:44.320000    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.237:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:08:44.320336 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:44.320070    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.237:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:08:44.650143 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:44.650113    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:45.005715 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.005623    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:45.282041 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.281954    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:45.373524 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.373490    4549 generic.go:356] "Generic (PLEG): container finished" podID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerID="41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125" exitCode=0
Oct 30 22:08:45.373727 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.373566    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" event={"ID":"05bbcc92-3e2a-46dd-bddc-4edf7c31629b","Type":"ContainerDied","Data":"41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"}
Oct 30 22:08:45.374029 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.374012    4549 scope.go:122] "RemoveContainer" containerID="41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"
Oct 30 22:08:45.375371 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.375335    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:45.375507 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.375429    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:45.376084 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.376039    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:45.425866 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.425815    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:45.570122 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:45.570080    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:46.399682 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:46.399647    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" event={"ID":"05bbcc92-3e2a-46dd-bddc-4edf7c31629b","Type":"ContainerStarted","Data":"8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0"}
Oct 30 22:08:46.594984 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:46.594939    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:46.725249 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:46.725153    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:46.942099 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:46.942059    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:46.942352 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:46.942327    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused" start-of-body=
Oct 30 22:08:46.942437 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:46.942402    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused"
Oct 30 22:08:47.945584 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:47.945527    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused" start-of-body=
Oct 30 22:08:47.946153 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:47.945650    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused"
Oct 30 22:08:48.260467 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:48.260379    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:48.941484 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:48.941443    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused" start-of-body=
Oct 30 22:08:48.941680 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:48.941543    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused"
Oct 30 22:08:49.942405 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:49.942367    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused" start-of-body=
Oct 30 22:08:49.942881 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:49.942420    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused"
Oct 30 22:08:50.034933 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:50.034897    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:50.941706 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:50.941665    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused" start-of-body=
Oct 30 22:08:50.941931 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:50.941737    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.237:15021/healthz/ready\": dial tcp 10.133.0.237:15021: connect: connection refused"
Oct 30 22:08:51.529202 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:51.529157    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:51.996849 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:51.996808    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:51.998019 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:51.997982    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:51.999928 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:51.999906    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:08:53.069620 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:53.069576    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:54.621671 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:54.621607    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:56.107802 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:56.107769    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:56.573889 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:08:56.573853    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:08:57.518298 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:57.518248    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:58.924659 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:08:58.924625    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:00.409515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:00.409480    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:01.707227 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:01.707190    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:03.355086 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:03.355051    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:04.814924 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:04.814796    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:06.304587 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:06.304549    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:07.726533 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:07.726493    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:09.127252 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:09.127218    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:10.620804 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:10.620763    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:12.007456 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:12.007418    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:12.888399 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:12.888361    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:13.429109 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:13.429070    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:14.923024 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:14.922982    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:16.361274 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:16.361205    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:17.860139 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:17.860105    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:19.427842 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:19.427788    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.915474 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:20.915432    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:22.301892 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:22.301851    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:23.825530 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:23.825497    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:25.327262 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:25.327219    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:26.573255 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:09:26.573219    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:09:26.818875 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:26.818817    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:27.933889 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:27.933852    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:28.532928 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:28.532894    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:30.155551 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:30.151155    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:31.710653 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:31.710613    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:32.954303 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:32.954269    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:33.105505 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:33.105472    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:34.658955 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:34.658912    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:36.007693 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:36.007651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:37.410426 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:37.410388    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:38.675089 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:38.675052    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:38.796064 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:38.796032    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:38.925692 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:38.925601    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:40.109150 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:40.109115    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:40.519474 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:40.519369    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:40.655289 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:40.655250    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:41.148423 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:41.148385    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:46.953222 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:46.953186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:47.102648 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:47.102615    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:47.247638 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:47.247558    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"]
Oct 30 22:09:47.247905 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:47.247873    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" containerID="cri-o://8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0" gracePeriod=30
Oct 30 22:09:47.309317 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:47.309281    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:51.596960 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:51.596925    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:52.000862 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.000744    4549 patch_prober.go:29] interesting pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:52.000862 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.000822    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:52.505899 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.505873    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:09:52.531449 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531368    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531449 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531407    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531449 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531426    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531731 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531491    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531731 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531527    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531731 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531656    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:52.531942 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531727    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531942 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531801    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531942 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531856    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531942 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531887    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo\") pod \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\" (UID: \"05bbcc92-3e2a-46dd-bddc-4edf7c31629b\") "
Oct 30 22:09:52.531942 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531894    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:52.532217 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.531989    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:52.532217 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532110    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:52.532349 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532228    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:52.532421 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532398    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.532421 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532418    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.532535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532431    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.532535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532442    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.532535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.532455    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.533818 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.533791    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:52.534148 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.534128    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:52.534293 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.534275    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:52.534358 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.534286    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5" pod "05bbcc92-3e2a-46dd-bddc-4edf7c31629b" (UID: "05bbcc92-3e2a-46dd-bddc-4edf7c31629b"). InnerVolumeSpecName "kube-api-access-kkcm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:52.633292 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.633238    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.633292 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.633286    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.633292 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.633296    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkcm5\" (UniqueName: \"kubernetes.io/projected/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-kube-api-access-kkcm5\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.633780 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.633306    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bbcc92-3e2a-46dd-bddc-4edf7c31629b-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:09:52.721306 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.721272    4549 generic.go:356] "Generic (PLEG): container finished" podID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerID="8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0" exitCode=0
Oct 30 22:09:52.721483 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.721315    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" event={"ID":"05bbcc92-3e2a-46dd-bddc-4edf7c31629b","Type":"ContainerDied","Data":"8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0"}
Oct 30 22:09:52.721483 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.721323    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"
Oct 30 22:09:52.721483 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.721384    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg" event={"ID":"05bbcc92-3e2a-46dd-bddc-4edf7c31629b","Type":"ContainerDied","Data":"95d4a133b522bffd6fd89864970b5f643a6f12dd2ec6f3d7d81af846f28c2421"}
Oct 30 22:09:52.721483 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.721406    4549 scope.go:122] "RemoveContainer" containerID="8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0"
Oct 30 22:09:52.731178 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.731152    4549 scope.go:122] "RemoveContainer" containerID="41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"
Oct 30 22:09:52.739286 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.739264    4549 scope.go:122] "RemoveContainer" containerID="8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0"
Oct 30 22:09:52.739577 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:09:52.739557    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0\": container with ID starting with 8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0 not found: ID does not exist" containerID="8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0"
Oct 30 22:09:52.739662 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.739591    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0"} err="failed to get container status \"8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0\": rpc error: code = NotFound desc = could not find container \"8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0\": container with ID starting with 8353075d2e2866446321c09c0eb4029682669e97c5069e1aec3d834ffa4248a0 not found: ID does not exist"
Oct 30 22:09:52.739662 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.739616    4549 scope.go:122] "RemoveContainer" containerID="41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"
Oct 30 22:09:52.739891 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:09:52.739872    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125\": container with ID starting with 41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125 not found: ID does not exist" containerID="41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"
Oct 30 22:09:52.739944 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.739899    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125"} err="failed to get container status \"41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125\": rpc error: code = NotFound desc = could not find container \"41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125\": container with ID starting with 41a5e2afe2be8557067bf57fe542dad98cb03e43c2a82e24a737fcfe315ba125 not found: ID does not exist"
Oct 30 22:09:52.771412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.771378    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"]
Oct 30 22:09:52.777104 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:52.777074    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg"]
Oct 30 22:09:53.341730 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:53.341695    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" path="/var/lib/kubelet/pods/05bbcc92-3e2a-46dd-bddc-4edf7c31629b/volumes"
Oct 30 22:09:53.843123 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:09:53.843086    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:56.572731 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:09:56.572694    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:10:08.486422 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:08.486342    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:09.113220 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:09.113185    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:10.627772 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:10.627737    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:13.079379 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:13.079302    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:18.949404 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:18.949356    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:10:18.989014 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:18.988989    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:10:20.960533 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:20.960500    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:21.074543 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:21.074515    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.809621 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:24.809579    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:26.574414 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:10:26.574374    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:10:28.956859 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:28.954735    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:36.451149 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:36.451109    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:37.995177 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:10:37.995141    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:56.572628 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:10:56.572586    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:11:13.226916 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:13.226875    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:25.352452 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.352403    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:25.812646 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.812547    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"]
Oct 30 22:11:25.813087 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813066    4549 cpu_manager.go:414] "containerMap: removing container" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy"
Oct 30 22:11:25.813218 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813089    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy"
Oct 30 22:11:25.813218 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813107    4549 cpu_manager.go:414] "containerMap: removing container" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy"
Oct 30 22:11:25.813218 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813114    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy"
Oct 30 22:11:25.813218 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813144    4549 cpu_manager.go:414] "containerMap: removing container" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy"
Oct 30 22:11:25.813218 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813152    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy"
Oct 30 22:11:25.813488 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813258    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="450bb602-3745-44b7-874e-4a64804eca48" containerName="istio-proxy"
Oct 30 22:11:25.813488 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813273    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy"
Oct 30 22:11:25.813488 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.813282    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="05bbcc92-3e2a-46dd-bddc-4edf7c31629b" containerName="istio-proxy"
Oct 30 22:11:25.816225 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.816199    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.840692 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.840666    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6ad3224d-openshift-default-dockercfg-r6pws\""
Oct 30 22:11:25.870743 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.870701    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"]
Oct 30 22:11:25.882034 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882002    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882185 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882060    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882185 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882161    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882265 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882207    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnfhn\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882265 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882231    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882265 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882256    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882376 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882284    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882431 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882401    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.882470 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.882450    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.898202 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.898169    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"]
Oct 30 22:11:25.901526 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.901505    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985269    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985331    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnfhn\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985369    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985397    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985446    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985500    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-maas-api-tls\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985531    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj7g9\" (UniqueName: \"kubernetes.io/projected/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-kube-api-access-xj7g9\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985557    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985581    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985605    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985642    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-metrics-certs\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985676    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-openshift-service-ca\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:25.985855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.985717    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:25.988790 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.988751    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.005290 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.989553    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.020044 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.989800    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.020772 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.990521    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.021055 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.990669    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.021397 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:25.992404    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"]
Oct 30 22:11:26.021397 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.002380    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:26.021397 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.005204    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.021397 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.016731    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.024647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.024622    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.028404 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.028145    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnfhn\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn\") pod \"e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") " pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.086614 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.086573    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-maas-api-tls\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.086882 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.086622    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj7g9\" (UniqueName: \"kubernetes.io/projected/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-kube-api-access-xj7g9\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.086882 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.086664    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-metrics-certs\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.086882 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.086694    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-openshift-service-ca\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.087469 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.087437    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-openshift-service-ca\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.089299 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.089278    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-maas-api-tls\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.089467 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.089446    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-metrics-certs\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.114449 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.114416    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj7g9\" (UniqueName: \"kubernetes.io/projected/5b9a4df4-c4fd-4b29-accd-535bac1e0f11-kube-api-access-xj7g9\") pod \"maas-api-54fbf74c89-8brn5\" (UID: \"5b9a4df4-c4fd-4b29-accd-535bac1e0f11\") " pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.133015 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.132989    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:26.200716 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.200682    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:26.213579 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.213547    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:26.444734 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.444704    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"]
Oct 30 22:11:26.448615 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:11:26.448587    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea160410_b2a5_4a89_b71c_a67a0e44ad77.slice/crio-f720fe1adb2aa5e846166c1ef3135da4064269f00c544da5d07a13a07983da72 WatchSource:0}: Error finding container f720fe1adb2aa5e846166c1ef3135da4064269f00c544da5d07a13a07983da72: Status 404 returned error can't find the container with id f720fe1adb2aa5e846166c1ef3135da4064269f00c544da5d07a13a07983da72
Oct 30 22:11:26.450494 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.450460    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:26.450603 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.450534    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:26.450603 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.450562    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:26.517464 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.517346    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"]
Oct 30 22:11:26.519753 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:11:26.519720    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9a4df4_c4fd_4b29_accd_535bac1e0f11.slice/crio-e689060c8851d31944424ec833cf787d9d84badce1417aedac7c50ffa06542e8 WatchSource:0}: Error finding container e689060c8851d31944424ec833cf787d9d84badce1417aedac7c50ffa06542e8: Status 404 returned error can't find the container with id e689060c8851d31944424ec833cf787d9d84badce1417aedac7c50ffa06542e8
Oct 30 22:11:26.521375 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:26.521353    4549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:11:26.574958 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:11:26.574914    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:11:27.329541 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.329502    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" event={"ID":"ea160410-b2a5-4a89-b71c-a67a0e44ad77","Type":"ContainerStarted","Data":"f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"}
Oct 30 22:11:27.329723 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.329566    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" event={"ID":"ea160410-b2a5-4a89-b71c-a67a0e44ad77","Type":"ContainerStarted","Data":"f720fe1adb2aa5e846166c1ef3135da4064269f00c544da5d07a13a07983da72"}
Oct 30 22:11:27.331126 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.331100    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5" event={"ID":"5b9a4df4-c4fd-4b29-accd-535bac1e0f11","Type":"ContainerStarted","Data":"2b6d6072f2beb1c711c271302094e7eb44011679fec013766d3b868b997f4a25"}
Oct 30 22:11:27.331262 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.331142    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5" event={"ID":"5b9a4df4-c4fd-4b29-accd-535bac1e0f11","Type":"ContainerStarted","Data":"e689060c8851d31944424ec833cf787d9d84badce1417aedac7c50ffa06542e8"}
Oct 30 22:11:27.331262 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.331174    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:27.386410 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.386350    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podStartSLOduration=2.3863317459999998 podStartE2EDuration="2.386331746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:27.384040089 +0000 UTC m=+4270.158440029" watchObservedRunningTime="2026-10-30 22:11:27.386331746 +0000 UTC m=+4270.160731686"
Oct 30 22:11:27.425405 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:27.425347    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5" podStartSLOduration=2.074254681 podStartE2EDuration="2.425333274s" totalImagesPullingTime="351.078593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:27.423811626 +0000 UTC m=+4270.198211565" watchObservedRunningTime="2026-10-30 22:11:27.425333274 +0000 UTC m=+4270.199733213"
Oct 30 22:11:28.133345 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:28.133307    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:28.138771 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:28.138734    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:28.340342 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:28.340309    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:28.341342 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:28.341319    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:29.030223 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:29.030192    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:30.877990 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:30.877957    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:31.906195 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:31.906158    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:32.071253 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:32.071223    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:32.212972 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:32.212873    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:33.346458 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:33.346427    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54fbf74c89-8brn5"
Oct 30 22:11:33.421885 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:33.421816    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"]
Oct 30 22:11:33.422124 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:33.422097    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" podUID="47b197df-f838-436e-9800-dab4962d7572" containerName="maas-api" containerID="cri-o://c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e" gracePeriod=30
Oct 30 22:11:33.633120 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:33.633012    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:34.016989 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.016886    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" podUID="47b197df-f838-436e-9800-dab4962d7572" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.30:8443/health\": dial tcp 10.133.0.30:8443: connect: connection refused"
Oct 30 22:11:34.670920 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.670894    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 22:11:34.796276 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.796189    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs\") pod \"47b197df-f838-436e-9800-dab4962d7572\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") "
Oct 30 22:11:34.796276 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.796228    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca\") pod \"47b197df-f838-436e-9800-dab4962d7572\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") "
Oct 30 22:11:34.796276 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.796259    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls\") pod \"47b197df-f838-436e-9800-dab4962d7572\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") "
Oct 30 22:11:34.796527 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.796293    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h\" (UniqueName: \"kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h\") pod \"47b197df-f838-436e-9800-dab4962d7572\" (UID: \"47b197df-f838-436e-9800-dab4962d7572\") "
Oct 30 22:11:34.796674 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.796645    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca" pod "47b197df-f838-436e-9800-dab4962d7572" (UID: "47b197df-f838-436e-9800-dab4962d7572"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:34.798676 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.798646    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls" pod "47b197df-f838-436e-9800-dab4962d7572" (UID: "47b197df-f838-436e-9800-dab4962d7572"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:34.798816 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.798685    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs" pod "47b197df-f838-436e-9800-dab4962d7572" (UID: "47b197df-f838-436e-9800-dab4962d7572"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:34.798816 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.798705    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h" pod "47b197df-f838-436e-9800-dab4962d7572" (UID: "47b197df-f838-436e-9800-dab4962d7572"). InnerVolumeSpecName "kube-api-access-tkm6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:34.898151 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.898103    4549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-metrics-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:11:34.898151 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.898144    4549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b197df-f838-436e-9800-dab4962d7572-openshift-service-ca\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:11:34.898151 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.898160    4549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b197df-f838-436e-9800-dab4962d7572-maas-api-tls\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:11:34.898421 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:34.898172    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkm6h\" (UniqueName: \"kubernetes.io/projected/47b197df-f838-436e-9800-dab4962d7572-kube-api-access-tkm6h\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:11:35.132645 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.132606    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:35.377607 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.377570    4549 generic.go:356] "Generic (PLEG): container finished" podID="47b197df-f838-436e-9800-dab4962d7572" containerID="c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e" exitCode=0
Oct 30 22:11:35.377815 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.377641    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" event={"ID":"47b197df-f838-436e-9800-dab4962d7572","Type":"ContainerDied","Data":"c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e"}
Oct 30 22:11:35.377815 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.377653    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"
Oct 30 22:11:35.377815 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.377694    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx" event={"ID":"47b197df-f838-436e-9800-dab4962d7572","Type":"ContainerDied","Data":"e0d3ca60ca7c698c10a7c67e3a4b8b5d9e0f7d27e2e98035945bbc73508185fb"}
Oct 30 22:11:35.377815 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.377715    4549 scope.go:122] "RemoveContainer" containerID="c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e"
Oct 30 22:11:35.388216 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.388196    4549 scope.go:122] "RemoveContainer" containerID="c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e"
Oct 30 22:11:35.388529 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:11:35.388508    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e\": container with ID starting with c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e not found: ID does not exist" containerID="c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e"
Oct 30 22:11:35.388598 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.388540    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e"} err="failed to get container status \"c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e\": rpc error: code = NotFound desc = could not find container \"c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e\": container with ID starting with c17d5e4e8974231d7682af3ae907e8f3d05612244f0f25c36e99a5711f014f9e not found: ID does not exist"
Oct 30 22:11:35.429231 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.429193    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"]
Oct 30 22:11:35.439976 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:35.439940    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b898bf6b9-2hlbx"]
Oct 30 22:11:36.728742 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:36.728707    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:37.342844 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:37.342797    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47b197df-f838-436e-9800-dab4962d7572" path="/var/lib/kubelet/pods/47b197df-f838-436e-9800-dab4962d7572/volumes"
Oct 30 22:11:38.255737 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:38.255700    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:39.753974 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:39.753939    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:41.329616 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:41.329579    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:42.863468 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:42.863426    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:44.381999 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:44.381955    4549 patch_prober.go:29] interesting pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.238:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:11:44.382496 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:44.382038    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.238:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:11:44.621102 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:44.621061    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:46.238009 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.237972    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:46.445350 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.445313    4549 generic.go:356] "Generic (PLEG): container finished" podID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerID="f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3" exitCode=0
Oct 30 22:11:46.445563 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.445361    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" event={"ID":"ea160410-b2a5-4a89-b71c-a67a0e44ad77","Type":"ContainerDied","Data":"f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"}
Oct 30 22:11:46.445820 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.445801    4549 scope.go:122] "RemoveContainer" containerID="f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"
Oct 30 22:11:46.447046 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.447016    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:46.447110 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.447085    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:46.447148 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:46.447118    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:47.452510 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:47.452474    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" event={"ID":"ea160410-b2a5-4a89-b71c-a67a0e44ad77","Type":"ContainerStarted","Data":"d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec"}
Oct 30 22:11:47.951209 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:47.951160    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:48.133515 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:48.133480    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:48.133805 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:48.133781    4549 patch_prober.go:29] interesting pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused" start-of-body=
Oct 30 22:11:48.133887 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:48.133857    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused"
Oct 30 22:11:49.133511 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:49.133481    4549 patch_prober.go:29] interesting pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused" start-of-body=
Oct 30 22:11:49.133904 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:49.133531    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused"
Oct 30 22:11:49.458527 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:49.458415    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:50.134257 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:50.134220    4549 patch_prober.go:29] interesting pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused" start-of-body=
Oct 30 22:11:50.134681 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:50.134283    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused"
Oct 30 22:11:50.936971 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:50.936920    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:51.133328 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:51.133291    4549 patch_prober.go:29] interesting pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused" start-of-body=
Oct 30 22:11:51.133525 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:51.133345    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused"
Oct 30 22:11:52.133904 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:52.133865    4549 patch_prober.go:29] interesting pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused" start-of-body=
Oct 30 22:11:52.134363 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:52.133929    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.238:15021/healthz/ready\": dial tcp 10.133.0.238:15021: connect: connection refused"
Oct 30 22:11:52.432434 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:52.432331    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:53.137960 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:53.137929    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:53.138463 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:53.138322    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:53.139161 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:53.139139    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:11:53.816213 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:53.816180    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:55.333224 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:55.333191    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:55.489360 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:55.489323    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:55.626349 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:55.626258    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:56.572523 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:11:56.572491    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:11:56.687903 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:56.687868    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:57.134172 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:57.134137    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:57.287992 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:11:57.287961    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:01.217577 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:01.217539    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"]
Oct 30 22:12:01.218525 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:01.218488    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy" containerID="cri-o://d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec" gracePeriod=30
Oct 30 22:12:01.283131 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:01.283088    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:02.593305 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:02.593271    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:02.975960 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:02.975873    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"]
Oct 30 22:12:02.976781 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:02.976751    4549 cpu_manager.go:414] "containerMap: removing container" podUID="47b197df-f838-436e-9800-dab4962d7572" containerName="maas-api"
Oct 30 22:12:02.976925 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:02.976784    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47b197df-f838-436e-9800-dab4962d7572" containerName="maas-api"
Oct 30 22:12:02.976980 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:02.976943    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="47b197df-f838-436e-9800-dab4962d7572" containerName="maas-api"
Oct 30 22:12:02.980650 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:02.980633    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.000739 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.000707    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"]
Oct 30 22:12:03.009582 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.009546    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-8c3a04-openshift-default-dockercfg-x96x9\""
Oct 30 22:12:03.046656 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.046616    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.046656 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.046655    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.046939 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.046682    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.046939 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.046822    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.046939 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.046874    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.047118 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.046962    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x7mc\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.047118 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.047000    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.047118 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.047019    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.047118 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.047034    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.051971 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.051940    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:03.147731 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147691    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.147731 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147734    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148005 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147775    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148005 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147848    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148005 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147888    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148005 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147954    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x7mc\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148005 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.147979    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148190 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148005    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148190 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148028    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148332 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148305    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148427 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148402    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148586 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148568    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148681 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148661    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.148927 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.148906    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.151059 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.151038    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.156355 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.156329    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.168009 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.167982    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.168113 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.167992    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x7mc\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc\") pod \"e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") " pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.294394 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.294304    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:03.333283 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.333138    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:03.628181 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.628154    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"]
Oct 30 22:12:03.629967 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:12:03.629941    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b48e9d_dced_467e_802a_b33343e37552.slice/crio-ae29d73ff0d1811f1f7032131057231dc16a4c15eef65ffd62695493bfb0a6c4 WatchSource:0}: Error finding container ae29d73ff0d1811f1f7032131057231dc16a4c15eef65ffd62695493bfb0a6c4: Status 404 returned error can't find the container with id ae29d73ff0d1811f1f7032131057231dc16a4c15eef65ffd62695493bfb0a6c4
Oct 30 22:12:03.632292 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.632261    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:03.632375 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.632325    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:03.632375 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:03.632354    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:04.543818 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:04.543779    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" event={"ID":"94b48e9d-dced-467e-802a-b33343e37552","Type":"ContainerStarted","Data":"ce56ba9f8d8d3dd9919056b2e36b31d7fe97d5dd919a17752f1686df2ff3debf"}
Oct 30 22:12:04.544033 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:04.543867    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" event={"ID":"94b48e9d-dced-467e-802a-b33343e37552","Type":"ContainerStarted","Data":"ae29d73ff0d1811f1f7032131057231dc16a4c15eef65ffd62695493bfb0a6c4"}
Oct 30 22:12:04.601967 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:04.601910    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podStartSLOduration=2.601895938 podStartE2EDuration="2.601895938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:04.599818361 +0000 UTC m=+4307.374218300" watchObservedRunningTime="2026-10-30 22:12:04.601895938 +0000 UTC m=+4307.376295876"
Oct 30 22:12:05.295707 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:05.295670    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:05.300558 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:05.300534    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:05.548540 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:05.548455    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:05.549529 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:05.549508    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:06.483715 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.483655    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:12:06.554717 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.554682    4549 generic.go:356] "Generic (PLEG): container finished" podID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerID="d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec" exitCode=0
Oct 30 22:12:06.554915 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.554771    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" event={"ID":"ea160410-b2a5-4a89-b71c-a67a0e44ad77","Type":"ContainerDied","Data":"d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec"}
Oct 30 22:12:06.554915 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.554789    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"
Oct 30 22:12:06.554915 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.554859    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx" event={"ID":"ea160410-b2a5-4a89-b71c-a67a0e44ad77","Type":"ContainerDied","Data":"f720fe1adb2aa5e846166c1ef3135da4064269f00c544da5d07a13a07983da72"}
Oct 30 22:12:06.554915 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.554894    4549 scope.go:122] "RemoveContainer" containerID="d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec"
Oct 30 22:12:06.564333 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.564311    4549 scope.go:122] "RemoveContainer" containerID="f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"
Oct 30 22:12:06.573572 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.573553    4549 scope.go:122] "RemoveContainer" containerID="d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec"
Oct 30 22:12:06.573943 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:12:06.573919    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec\": container with ID starting with d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec not found: ID does not exist" containerID="d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec"
Oct 30 22:12:06.574018 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.573954    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec"} err="failed to get container status \"d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec\": rpc error: code = NotFound desc = could not find container \"d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec\": container with ID starting with d7b0191d1324dae24e5e3382583b8aeed631e18432a42b2dec12ddb7feaca3ec not found: ID does not exist"
Oct 30 22:12:06.574018 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.573973    4549 scope.go:122] "RemoveContainer" containerID="f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"
Oct 30 22:12:06.574227 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:12:06.574210    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3\": container with ID starting with f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3 not found: ID does not exist" containerID="f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"
Oct 30 22:12:06.574270 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.574233    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3"} err="failed to get container status \"f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3\": rpc error: code = NotFound desc = could not find container \"f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3\": container with ID starting with f5b03c7d205619644155ee672620f00317f6f7a89c4d24c036370972e549e8b3 not found: ID does not exist"
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.583996    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584062    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584094    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584170    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584228    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584273    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584322    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584367    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584398    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs\") pod \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\" (UID: \"ea160410-b2a5-4a89-b71c-a67a0e44ad77\") "
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584576    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584777    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584791    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.584793    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.585450    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.585512    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.587956    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.588009    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.588647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.588460    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "kube-api-access-bnfhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.589535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.588911    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo" pod "ea160410-b2a5-4a89-b71c-a67a0e44ad77" (UID: "ea160410-b2a5-4a89-b71c-a67a0e44ad77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:06.685431 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685397    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685431 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685426    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685431 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685436    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685685 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685446    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnfhn\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-kube-api-access-bnfhn\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685685 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685455    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685685 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685465    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685685 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685472    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea160410-b2a5-4a89-b71c-a67a0e44ad77-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.685685 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.685482    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea160410-b2a5-4a89-b71c-a67a0e44ad77-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:12:06.899823 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.899789    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"]
Oct 30 22:12:06.911733 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:06.911698    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx"]
Oct 30 22:12:07.025846 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:07.025801    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:07.166674 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:07.166580    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:07.348802 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:07.348762    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" path="/var/lib/kubelet/pods/ea160410-b2a5-4a89-b71c-a67a0e44ad77/volumes"
Oct 30 22:12:08.491158 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.491105    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:08.660115 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.660081    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:08.823944 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.823894    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"]
Oct 30 22:12:08.824371 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.824354    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy"
Oct 30 22:12:08.824425 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.824373    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy"
Oct 30 22:12:08.824478 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.824468    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy"
Oct 30 22:12:08.828848 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.828807    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:08.839265 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.839238    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:08.846604 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.846584    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-8c3a04-dockercfg-6n42t\""
Oct 30 22:12:08.861757 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.861721    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"]
Oct 30 22:12:08.880066 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.880031    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-8c3a04\""
Oct 30 22:12:08.909558 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.909509    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:08.909773 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:08.909636    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55wsz\" (UniqueName: \"kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz\") pod \"payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.010785 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.010739    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55wsz\" (UniqueName: \"kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz\") pod \"payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.011016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.010866    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.011531 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.011511    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.035259 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.035226    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55wsz\" (UniqueName: \"kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz\") pod \"payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.142089 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.141999    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.463785 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.463683    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"]
Oct 30 22:12:09.469178 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:12:09.469145    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab652b7f_5236_4c44_bc47_ee431b6f35c3.slice/crio-ebd24fa75d53c8d46b0a03c3499b9a749f330e8be4e3e4408d42601a39a8e721 WatchSource:0}: Error finding container ebd24fa75d53c8d46b0a03c3499b9a749f330e8be4e3e4408d42601a39a8e721: Status 404 returned error can't find the container with id ebd24fa75d53c8d46b0a03c3499b9a749f330e8be4e3e4408d42601a39a8e721
Oct 30 22:12:09.571983 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.571947    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj" event={"ID":"ab652b7f-5236-4c44-bc47-ee431b6f35c3","Type":"ContainerStarted","Data":"78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362"}
Oct 30 22:12:09.572463 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.572001    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj" event={"ID":"ab652b7f-5236-4c44-bc47-ee431b6f35c3","Type":"ContainerStarted","Data":"ebd24fa75d53c8d46b0a03c3499b9a749f330e8be4e3e4408d42601a39a8e721"}
Oct 30 22:12:09.572463 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.572042    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:09.610666 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:09.610610    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj" podStartSLOduration=1.6105974920000001 podStartE2EDuration="1.610597492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:09.609415488 +0000 UTC m=+4312.383815429" watchObservedRunningTime="2026-10-30 22:12:09.610597492 +0000 UTC m=+4312.384997431"
Oct 30 22:12:10.112140 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:10.112104    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:11.623269 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:11.623231    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:13.048404 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:13.048365    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:13.333500 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:13.333460    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:14.553860 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:14.553778    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:16.053945 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:16.053911    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:17.647823 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:17.647785    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:19.167306 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:19.167248    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:20.580170 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:20.580135    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:12:20.713803 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:20.713771    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:21.590053 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:21.590014    4549 patch_prober.go:29] interesting pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.240:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:12:21.590486 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:21.590078    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.240:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:22.078064 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.078032    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:22.243800 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:12:22.243767    4549 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@38-57105-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b48e9d_dced_467e_802a_b33343e37552.slice/crio-ce56ba9f8d8d3dd9919056b2e36b31d7fe97d5dd919a17752f1686df2ff3debf.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:12:22.315419 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.315374    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:22.646949 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.646914    4549 generic.go:356] "Generic (PLEG): container finished" podID="94b48e9d-dced-467e-802a-b33343e37552" containerID="ce56ba9f8d8d3dd9919056b2e36b31d7fe97d5dd919a17752f1686df2ff3debf" exitCode=0
Oct 30 22:12:22.647398 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.646996    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" event={"ID":"94b48e9d-dced-467e-802a-b33343e37552","Type":"ContainerDied","Data":"ce56ba9f8d8d3dd9919056b2e36b31d7fe97d5dd919a17752f1686df2ff3debf"}
Oct 30 22:12:22.647460 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.647425    4549 scope.go:122] "RemoveContainer" containerID="ce56ba9f8d8d3dd9919056b2e36b31d7fe97d5dd919a17752f1686df2ff3debf"
Oct 30 22:12:22.648613 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.648586    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:22.648680 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.648654    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:22.648720 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:22.648685    4549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:23.661171 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:23.661136    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" event={"ID":"94b48e9d-dced-467e-802a-b33343e37552","Type":"ContainerStarted","Data":"a1ae9c59bab64e5cfd4ad96b8c8d6b0bcf179fd12e98ab490b1982dde7a813d7"}
Oct 30 22:12:23.752439 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:23.752401    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:24.295206 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:24.295158    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:24.295435 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:24.295358    4549 patch_prober.go:29] interesting pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused" start-of-body=
Oct 30 22:12:24.295435 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:24.295409    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused"
Oct 30 22:12:25.295311 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:25.295261    4549 patch_prober.go:29] interesting pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused" start-of-body=
Oct 30 22:12:25.295762 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:25.295341    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused"
Oct 30 22:12:25.336813 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:25.336778    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:26.295282 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:26.295244    4549 patch_prober.go:29] interesting pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused" start-of-body=
Oct 30 22:12:26.295499 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:26.295304    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused"
Oct 30 22:12:26.602106 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:12:26.602065    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:12:26.884962 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:26.883658    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:27.296810 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:27.296695    4549 patch_prober.go:29] interesting pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused" start-of-body=
Oct 30 22:12:27.296810 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:27.296773    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused"
Oct 30 22:12:28.295315 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:28.295279    4549 patch_prober.go:29] interesting pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused" start-of-body=
Oct 30 22:12:28.295531 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:28.295344    4549 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.240:15021/healthz/ready\": dial tcp 10.133.0.240:15021: connect: connection refused"
Oct 30 22:12:28.346326 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:28.346280    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:29.298882 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:29.298851    4549 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:29.299254 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:29.299229    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:29.299799 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:29.299777    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:12:29.832771 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:29.832733    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:31.227032 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:31.226986    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:32.690600 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:32.690566    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:34.127457 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:34.127421    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:35.788960 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:35.788920    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:37.227901 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:37.227866    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:38.614884 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:38.614852    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:40.005971 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:40.005935    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:41.458793 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:41.458735    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:42.990973 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:42.990929    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:44.464300 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:44.464269    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:45.869538 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:45.869504    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:47.450619 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:47.450373    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:47.784596 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:47.784504    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:48.924890 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:48.924823    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:50.317993 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:50.317963    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:51.727217 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:51.727179    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:53.242139 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:53.242106    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:54.626438 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:54.626402    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:56.014552 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:56.014508    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:56.573025 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:12:56.572992    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:12:57.520120 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:57.520079    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:59.029400 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:59.029309    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:59.399058 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:12:59.399006    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:00.427048 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:00.427015    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:02.020349 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:02.020311    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:03.476166 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:03.476125    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:04.910327 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:04.910291    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:06.323555 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:06.323524    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:07.724264 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:07.724226    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:08.275891 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:08.275853    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:09.218253 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:09.218221    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:10.613359 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:10.613324    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:11.996692 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:11.996651    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:13.437633 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:13.437597    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:13.566144 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:13.566112    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:14.918226 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:14.918190    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:16.307371 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:16.307334    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:17.729601 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:17.729562    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:19.207869 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:19.207813    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:20.610974 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:20.610943    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:20.751433 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:20.751404    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:22.019960 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:22.019920    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:23.410563 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:23.410530    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:24.810558 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:24.810525    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:26.198916 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:26.198884    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:26.573802 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:13:26.573761    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:13:27.647968 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:27.647929    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:29.023622 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:29.023586    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:30.428336 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:30.428301    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:31.816586 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:31.816548    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:33.222478 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:33.222446    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:34.628233 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:34.628197    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:36.106654 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:36.106619    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:37.614910 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:37.614874    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:39.006644 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:39.006608    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:40.427238 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:40.427188    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:41.819425 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:41.819390    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:43.398047 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:43.398014    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:44.884987 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:44.884943    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:46.399030 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:46.399000    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:47.814847 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:47.814798    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:49.218138 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:49.218103    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:50.807229 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:50.807186    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:52.216726 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:52.216690    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:52.676181 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:52.676146    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"]
Oct 30 22:13:52.801818 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:52.801789    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:52.930844 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:52.930736    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:56.574056 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:13:56.574010    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:13:57.676766 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:57.676713    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj" podUID="ab652b7f-5236-4c44-bc47-ee431b6f35c3" containerName="payload-pre-processing" containerID="cri-o://78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362" gracePeriod=25
Oct 30 22:13:57.932348 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:57.932274    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:13:57.973970 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:57.973909    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz\" (UniqueName: \"kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz\") pod \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") "
Oct 30 22:13:57.974191 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:57.974046    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume\") pod \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\" (UID: \"ab652b7f-5236-4c44-bc47-ee431b6f35c3\") "
Oct 30 22:13:57.974477 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:57.974447    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume" pod "ab652b7f-5236-4c44-bc47-ee431b6f35c3" (UID: "ab652b7f-5236-4c44-bc47-ee431b6f35c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:57.976497 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:57.976469    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz" pod "ab652b7f-5236-4c44-bc47-ee431b6f35c3" (UID: "ab652b7f-5236-4c44-bc47-ee431b6f35c3"). InnerVolumeSpecName "kube-api-access-55wsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:58.075121 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.075066    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55wsz\" (UniqueName: \"kubernetes.io/projected/ab652b7f-5236-4c44-bc47-ee431b6f35c3-kube-api-access-55wsz\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:13:58.075121 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.075114    4549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab652b7f-5236-4c44-bc47-ee431b6f35c3-plugins-config-volume\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:13:58.104730 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.104693    4549 generic.go:356] "Generic (PLEG): container finished" podID="ab652b7f-5236-4c44-bc47-ee431b6f35c3" containerID="78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362" exitCode=0
Oct 30 22:13:58.104963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.104740    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"
Oct 30 22:13:58.104963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.104762    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj" event={"ID":"ab652b7f-5236-4c44-bc47-ee431b6f35c3","Type":"ContainerDied","Data":"78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362"}
Oct 30 22:13:58.104963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.104808    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj" event={"ID":"ab652b7f-5236-4c44-bc47-ee431b6f35c3","Type":"ContainerDied","Data":"ebd24fa75d53c8d46b0a03c3499b9a749f330e8be4e3e4408d42601a39a8e721"}
Oct 30 22:13:58.104963 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.104865    4549 scope.go:122] "RemoveContainer" containerID="78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362"
Oct 30 22:13:58.114723 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.114701    4549 scope.go:122] "RemoveContainer" containerID="78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362"
Oct 30 22:13:58.115093 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:13:58.115064    4549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362\": container with ID starting with 78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362 not found: ID does not exist" containerID="78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362"
Oct 30 22:13:58.115205 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.115103    4549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362"} err="failed to get container status \"78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362\": rpc error: code = NotFound desc = could not find container \"78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362\": container with ID starting with 78e7e0ac5aae743dacafa0dc398c5321a96d90e6faae0b848df488dca77ec362 not found: ID does not exist"
Oct 30 22:13:58.155081 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.155045    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"]
Oct 30 22:13:58.159294 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.159260    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj"]
Oct 30 22:13:58.570538 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.570497    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:58.944675 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:58.944571    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:59.075153 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:59.074947    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:59.343338 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:59.343306    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab652b7f-5236-4c44-bc47-ee431b6f35c3" path="/var/lib/kubelet/pods/ab652b7f-5236-4c44-bc47-ee431b6f35c3/volumes"
Oct 30 22:13:59.627675 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:13:59.627587    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:05.573120 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:05.573086    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:05.704668 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:05.704629    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:05.849535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:05.849442    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"]
Oct 30 22:14:05.849738 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:05.849711    4549 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy" containerID="cri-o://a1ae9c59bab64e5cfd4ad96b8c8d6b0bcf179fd12e98ab490b1982dde7a813d7" gracePeriod=30
Oct 30 22:14:05.965543 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:05.965503    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:11.183332 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.182886    4549 generic.go:356] "Generic (PLEG): container finished" podID="94b48e9d-dced-467e-802a-b33343e37552" containerID="a1ae9c59bab64e5cfd4ad96b8c8d6b0bcf179fd12e98ab490b1982dde7a813d7" exitCode=0
Oct 30 22:14:11.183332 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.183054    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" event={"ID":"94b48e9d-dced-467e-802a-b33343e37552","Type":"ContainerDied","Data":"a1ae9c59bab64e5cfd4ad96b8c8d6b0bcf179fd12e98ab490b1982dde7a813d7"}
Oct 30 22:14:11.183332 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.183155    4549 scope.go:122] "RemoveContainer" containerID="ce56ba9f8d8d3dd9919056b2e36b31d7fe97d5dd919a17752f1686df2ff3debf"
Oct 30 22:14:11.628855 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.628033    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:14:11.766412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766319    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766382    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766640 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766464    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766640 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766520    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766640 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766560    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766850 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766648    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766850 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766683    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766850 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766705    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:11.766850 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766716    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.766850 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766841    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:11.767161 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.766878    4549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert\") pod \"94b48e9d-dced-467e-802a-b33343e37552\" (UID: \"94b48e9d-dced-467e-802a-b33343e37552\") "
Oct 30 22:14:11.767236 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.767168    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:11.767340 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.767320    4549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-certs\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.767412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.767347    4549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-workload-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.767412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.767361    4549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-data\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.767412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.767383    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:11.767412 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.767043    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:11.769305 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.769275    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "kube-api-access-5x7mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:11.769750 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.769730    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:11.769933 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.769911    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:11.792215 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.792179    4549 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo" pod "94b48e9d-dced-467e-802a-b33343e37552" (UID: "94b48e9d-dced-467e-802a-b33343e37552"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:14:11.868693 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.868662    4549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x7mc\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-kube-api-access-5x7mc\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.868693 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.868697    4549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-istio-envoy\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.868930 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.868708    4549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b48e9d-dced-467e-802a-b33343e37552-istiod-ca-cert\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.868930 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.868716    4549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b48e9d-dced-467e-802a-b33343e37552-credential-socket\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.868930 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.868725    4549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b48e9d-dced-467e-802a-b33343e37552-istio-podinfo\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:11.868930 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:11.868733    4549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b48e9d-dced-467e-802a-b33343e37552-istio-token\") on node \"ip-10-0-140-179.ec2.internal\" DevicePath \"\""
Oct 30 22:14:12.188790 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:12.188762    4549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"
Oct 30 22:14:12.189303 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:12.188809    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx" event={"ID":"94b48e9d-dced-467e-802a-b33343e37552","Type":"ContainerDied","Data":"ae29d73ff0d1811f1f7032131057231dc16a4c15eef65ffd62695493bfb0a6c4"}
Oct 30 22:14:12.189303 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:12.188886    4549 scope.go:122] "RemoveContainer" containerID="a1ae9c59bab64e5cfd4ad96b8c8d6b0bcf179fd12e98ab490b1982dde7a813d7"
Oct 30 22:14:12.258133 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:12.258106    4549 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"]
Oct 30 22:14:12.263193 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:12.263167    4549 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx"]
Oct 30 22:14:13.341887 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:13.341856    4549 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94b48e9d-dced-467e-802a-b33343e37552" path="/var/lib/kubelet/pods/94b48e9d-dced-467e-802a-b33343e37552/volumes"
Oct 30 22:14:13.583875 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:13.583823    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.572717 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:14:26.572681    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:14:40.149917 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:40.148121    4549 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:14:40.149917 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:40.148236    4549 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx" podUID="579c215d-3fff-4eba-8e6b-52b2054a8d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.234:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:14:51.278169 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:14:51.278131    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:56.573475 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:14:56.573435    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:15:13.683444 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:13.683408    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:14.145435 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:14.145397    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-654c5dd44b-gw588_7cfb0755-61bd-4864-bc81-d79fccb5e66c/manager/0.log"
Oct 30 22:15:14.856037 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:14.856004    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-tmsm2_74ff306c-2ab2-4ea5-8817-987e010342fc/proxy/0.log"
Oct 30 22:15:15.592945 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:15.592914    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54fbf74c89-8brn5_5b9a4df4-c4fd-4b29-accd-535bac1e0f11/maas-api/0.log"
Oct 30 22:15:17.322089 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:17.322051    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-9qhzp_d92ef50b-4758-42be-9b06-21b319ec61a8/postgres/0.log"
Oct 30 22:15:18.974518 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:18.974486    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:15:19.016226 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:19.016203    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d44wt_dabeb57c-4fde-4a6d-a4c8-e1943641d0aa/ovn-acl-logging/0.log"
Oct 30 22:15:21.226571 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:21.226544    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-jggkz_a2272918-faed-4465-b14f-19a53468a454/discovery/0.log"
Oct 30 22:15:21.458254 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:21.458219    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5649f7bc66-klvr2_d69f0413-b858-4239-8aeb-5b2e6b27b1d6/kube-auth-proxy/0.log"
Oct 30 22:15:22.231276 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:22.231244    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-h2mcv_2822d220-08fd-45d5-be5c-9ffa403c0f7e/istio-proxy/0.log"
Oct 30 22:15:22.435957 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:22.435914    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-nb8vx_579c215d-3fff-4eba-8e6b-52b2054a8d0c/istio-proxy/0.log"
Oct 30 22:15:22.836015 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:22.835982    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-p66fx_20d5e344-c230-4e7e-a922-e7868c0038d4/istio-proxy/0.log"
Oct 30 22:15:23.036005 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:23.035975    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-pnzz7_98b392c7-af0b-4e20-bf43-dbd024ba92b4/istio-proxy/0.log"
Oct 30 22:15:24.003740 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:24.003704    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-tbwcl_992ca47a-dcf6-4284-a77f-34af9342e718/payload-pre-processing/0.log"
Oct 30 22:15:24.188874 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:24.188817    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:15:25.382699 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:25.382671    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7_d1154792-d0fb-4c0c-9b69-75ff2b0905a4/main/0.log"
Oct 30 22:15:25.584551 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:25.584523    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-cdl4t_fd4d565e-8ace-4a8a-a8fb-114a90e3f64c/main/0.log"
Oct 30 22:15:25.769734 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:25.769659    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-4wgct_569e7b40-df79-438b-a350-7f4cf8202c9c/main/0.log"
Oct 30 22:15:26.346550 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:26.346511    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-c5dtn_0f292b66-399c-4566-8921-3f04abdaed5a/main/0.log"
Oct 30 22:15:26.572285 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:15:26.572244    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:15:26.727933 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:26.727863    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4_4bf73f9c-8107-47f1-9533-2aa3fa3e7606/main/0.log"
Oct 30 22:15:37.341899 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:37.341870    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-654c5dd44b-gw588_7cfb0755-61bd-4864-bc81-d79fccb5e66c/manager/0.log"
Oct 30 22:15:37.963580 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:37.963545    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-tmsm2_74ff306c-2ab2-4ea5-8817-987e010342fc/proxy/0.log"
Oct 30 22:15:38.480944 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:38.480914    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54fbf74c89-8brn5_5b9a4df4-c4fd-4b29-accd-535bac1e0f11/maas-api/0.log"
Oct 30 22:15:39.841951 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:39.841921    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-9qhzp_d92ef50b-4758-42be-9b06-21b319ec61a8/postgres/0.log"
Oct 30 22:15:42.948155 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:42.948118    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-jggkz_a2272918-faed-4465-b14f-19a53468a454/discovery/0.log"
Oct 30 22:15:43.174534 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:43.174501    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5649f7bc66-klvr2_d69f0413-b858-4239-8aeb-5b2e6b27b1d6/kube-auth-proxy/0.log"
Oct 30 22:15:43.950710 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:43.950676    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-h2mcv_2822d220-08fd-45d5-be5c-9ffa403c0f7e/istio-proxy/0.log"
Oct 30 22:15:44.172358 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:44.172327    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-nb8vx_579c215d-3fff-4eba-8e6b-52b2054a8d0c/istio-proxy/0.log"
Oct 30 22:15:44.574707 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:44.574676    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-p66fx_20d5e344-c230-4e7e-a922-e7868c0038d4/istio-proxy/0.log"
Oct 30 22:15:44.777208 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:44.777172    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-pnzz7_98b392c7-af0b-4e20-bf43-dbd024ba92b4/istio-proxy/0.log"
Oct 30 22:15:45.747704 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:45.747679    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-tbwcl_992ca47a-dcf6-4284-a77f-34af9342e718/payload-pre-processing/0.log"
Oct 30 22:15:45.925307 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:45.925270    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:15:46.668723 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:46.668692    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-j2pl7_d1154792-d0fb-4c0c-9b69-75ff2b0905a4/main/0.log"
Oct 30 22:15:46.859711 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:46.859679    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-cdl4t_fd4d565e-8ace-4a8a-a8fb-114a90e3f64c/main/0.log"
Oct 30 22:15:47.052373 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:47.052277    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-4wgct_569e7b40-df79-438b-a350-7f4cf8202c9c/main/0.log"
Oct 30 22:15:47.653355 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:47.653328    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-c5dtn_0f292b66-399c-4566-8921-3f04abdaed5a/main/0.log"
Oct 30 22:15:48.052275 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:15:48.052185    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-zt2j4_4bf73f9c-8107-47f1-9533-2aa3fa3e7606/main/0.log"
Oct 30 22:15:56.572667 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:15:56.572629    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:16:13.807871 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:16:13.807791    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:21.294605 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:16:21.294578    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-654c5dd44b-gw588_7cfb0755-61bd-4864-bc81-d79fccb5e66c/manager/0.log"
Oct 30 22:16:21.593051 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:16:21.593021    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-654c5dd44b-gw588_7cfb0755-61bd-4864-bc81-d79fccb5e66c/manager/0.log"
Oct 30 22:16:22.319792 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:16:22.319764    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:16:22.495118 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:16:22.495090    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:16:26.572868 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:16:26.572814    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:16:56.572996 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:16:56.572956    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:17:03.945526 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:03.945496    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-654c5dd44b-gw588_7cfb0755-61bd-4864-bc81-d79fccb5e66c/manager/0.log"
Oct 30 22:17:04.257180 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:04.257098    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-654c5dd44b-gw588_7cfb0755-61bd-4864-bc81-d79fccb5e66c/manager/0.log"
Oct 30 22:17:04.990432 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:04.990403    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:17:05.177212 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:05.177184    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:17:05.555263 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:05.555220    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:06.707107 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:06.707070    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:10.075276 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:10.075249    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cpdvw_4c1d2b37-38ac-4aca-90d3-144d6537006f/global-pull-secret-syncer/0.log"
Oct 30 22:17:11.048733 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:11.048704    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log"
Oct 30 22:17:11.635535 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:11.635501    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-179.ec2.internal_0d94f708ebb6ce685e97c708d7e6cc9d/haproxy/0.log"
Oct 30 22:17:13.971589 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:13.971552    4549 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8t9kn_81eb3986-da71-4e0e-862b-c84d3f33815c/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:20.276647 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.276608    4549 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"]
Oct 30 22:17:20.277066 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277020    4549 cpu_manager.go:414] "containerMap: removing container" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy"
Oct 30 22:17:20.277066 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277031    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy"
Oct 30 22:17:20.277066 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277044    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ab652b7f-5236-4c44-bc47-ee431b6f35c3" containerName="payload-pre-processing"
Oct 30 22:17:20.277066 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277053    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab652b7f-5236-4c44-bc47-ee431b6f35c3" containerName="payload-pre-processing"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277074    4549 cpu_manager.go:414] "containerMap: removing container" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277081    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277096    4549 cpu_manager.go:414] "containerMap: removing container" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277101    4549 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277164    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277173    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab652b7f-5236-4c44-bc47-ee431b6f35c3" containerName="payload-pre-processing"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277179    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea160410-b2a5-4a89-b71c-a67a0e44ad77" containerName="istio-proxy"
Oct 30 22:17:20.277309 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.277186    4549 memory_manager.go:365] "RemoveStaleState removing state" podUID="94b48e9d-dced-467e-802a-b33343e37552" containerName="istio-proxy"
Oct 30 22:17:20.280152 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.280137    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.298694 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.298663    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"]
Oct 30 22:17:20.339874 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.339817    4549 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-47t7f\"/\"default-dockercfg-5hv7n\""
Oct 30 22:17:20.340033 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.340003    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsmfl\" (UniqueName: \"kubernetes.io/projected/19985f7b-d4da-4633-bcfc-fd8caaed2af8-kube-api-access-bsmfl\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.340079 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.340046    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-proc\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.340079 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.340073    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-sys\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.340219 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.340206    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-podres\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.340262 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.340244    4549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-lib-modules\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.388526 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.388496    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47t7f\"/\"openshift-service-ca.crt\""
Oct 30 22:17:20.388694 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.388534    4549 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47t7f\"/\"kube-root-ca.crt\""
Oct 30 22:17:20.440814 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440785    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-proc\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440851    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-sys\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440903    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-podres\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440915    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-proc\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440923    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-lib-modules\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440987    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-sys\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.441000    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-podres\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441016 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.441012    4549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsmfl\" (UniqueName: \"kubernetes.io/projected/19985f7b-d4da-4633-bcfc-fd8caaed2af8-kube-api-access-bsmfl\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.441615 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.440995    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19985f7b-d4da-4633-bcfc-fd8caaed2af8-lib-modules\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.461351 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.461319    4549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsmfl\" (UniqueName: \"kubernetes.io/projected/19985f7b-d4da-4633-bcfc-fd8caaed2af8-kube-api-access-bsmfl\") pod \"perf-node-gather-daemonset-bss86\" (UID: \"19985f7b-d4da-4633-bcfc-fd8caaed2af8\") " pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.592930 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.592895    4549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:20.892966 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:20.892938    4549 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"]
Oct 30 22:17:20.895709 ip-10-0-140-179 kubenswrapper[4549]: W1030 22:17:20.895681    4549 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod19985f7b_d4da_4633_bcfc_fd8caaed2af8.slice/crio-a691716395b7dc2ce0fec904ccfa38177f9bacf7d492f3191e55b10e14f0affa WatchSource:0}: Error finding container a691716395b7dc2ce0fec904ccfa38177f9bacf7d492f3191e55b10e14f0affa: Status 404 returned error can't find the container with id a691716395b7dc2ce0fec904ccfa38177f9bacf7d492f3191e55b10e14f0affa
Oct 30 22:17:21.137264 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:21.137231    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86" event={"ID":"19985f7b-d4da-4633-bcfc-fd8caaed2af8","Type":"ContainerStarted","Data":"2847f5f778f154e0de4133509106292acd4d0096c82a039641a9f5c18a10314b"}
Oct 30 22:17:21.137466 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:21.137284    4549 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86" event={"ID":"19985f7b-d4da-4633-bcfc-fd8caaed2af8","Type":"ContainerStarted","Data":"a691716395b7dc2ce0fec904ccfa38177f9bacf7d492f3191e55b10e14f0affa"}
Oct 30 22:17:21.137466 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:21.137315    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:21.183519 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:21.183415    4549 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86" podStartSLOduration=1.183400668 podStartE2EDuration="1.183400668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:21.182227479 +0000 UTC m=+4623.956627418" watchObservedRunningTime="2026-10-30 22:17:21.183400668 +0000 UTC m=+4623.957800607"
Oct 30 22:17:22.227601 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:22.227472    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2g6cr_1a57b7a0-984a-4b56-96af-ba5144c9c7d4/node-exporter/0.log"
Oct 30 22:17:22.361380 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:22.361355    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2g6cr_1a57b7a0-984a-4b56-96af-ba5144c9c7d4/kube-rbac-proxy/0.log"
Oct 30 22:17:22.490315 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:22.490229    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2g6cr_1a57b7a0-984a-4b56-96af-ba5144c9c7d4/init-textfile/0.log"
Oct 30 22:17:24.900619 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:24.900588    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-d5lhp_726bbf8e-ffe8-4f24-8600-645b2ba16089/prometheus-operator-admission-webhook/0.log"
Oct 30 22:17:25.038893 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:25.038864    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86c86766b4-2gwt9_277cb4db-e7d5-4b31-ba96-1b6de72438dc/telemeter-client/0.log"
Oct 30 22:17:25.166956 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:25.166885    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86c86766b4-2gwt9_277cb4db-e7d5-4b31-ba96-1b6de72438dc/reload/0.log"
Oct 30 22:17:25.292994 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:25.292963    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86c86766b4-2gwt9_277cb4db-e7d5-4b31-ba96-1b6de72438dc/kube-rbac-proxy/0.log"
Oct 30 22:17:26.573301 ip-10-0-140-179 kubenswrapper[4549]: E1030 22:17:26.573263    4549 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5blgd" containerName="extractor"
Oct 30 22:17:27.151749 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:27.151720    4549 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-47t7f/perf-node-gather-daemonset-bss86"
Oct 30 22:17:29.718531 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:29.718456    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79f47bdf55-s7dmf_ed221151-cbc4-4d2f-be7e-b38e35e6f164/console/0.log"
Oct 30 22:17:29.861064 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:29.861029    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-ccg5n_1987d310-712c-4d63-b06c-9ac7aa3f961c/download-server/0.log"
Oct 30 22:17:33.718931 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:33.718891    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-jggkz_a2272918-faed-4465-b14f-19a53468a454/discovery/0.log"
Oct 30 22:17:33.755312 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:33.755281    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5649f7bc66-klvr2_d69f0413-b858-4239-8aeb-5b2e6b27b1d6/kube-auth-proxy/0.log"
Oct 30 22:17:34.014460 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.014362    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h4cm4_e5a2f410-97fd-4998-962c-dd2f8cd3281d/dns/0.log"
Oct 30 22:17:34.014841 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.014808    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-h2mcv_2822d220-08fd-45d5-be5c-9ffa403c0f7e/istio-proxy/0.log"
Oct 30 22:17:34.085147 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.085110    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-nb8vx_579c215d-3fff-4eba-8e6b-52b2054a8d0c/istio-proxy/0.log"
Oct 30 22:17:34.139691 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.139660    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h4cm4_e5a2f410-97fd-4998-962c-dd2f8cd3281d/kube-rbac-proxy/0.log"
Oct 30 22:17:34.221150 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.221122    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-p66fx_20d5e344-c230-4e7e-a922-e7868c0038d4/istio-proxy/0.log"
Oct 30 22:17:34.287312 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.287228    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-pnzz7_98b392c7-af0b-4e20-bf43-dbd024ba92b4/istio-proxy/0.log"
Oct 30 22:17:34.394287 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.394259    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wsmb_3b759b7b-3f86-427d-a1f7-6382fbe96464/dns-node-resolver/0.log"
Oct 30 22:17:34.592196 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.592166    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-tbwcl_992ca47a-dcf6-4284-a77f-34af9342e718/payload-pre-processing/0.log"
Oct 30 22:17:34.658484 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:34.658446    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-kjg7p_a8254b5c-1d56-4b59-8760-0d16e3761711/payload-processing/0.log"
Oct 30 22:17:35.663660 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:35.663629    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7755d75f65-wh86x_37a58eec-af4e-45a4-b6ad-a1542049b08c/registry/0.log"
Oct 30 22:17:35.900346 ip-10-0-140-179 kubenswrapper[4549]: I1030 22:17:35.900314    4549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lb6cv_b346fb18-266d-44b4-998e-afdaf5ae0085/node-ca/0.log"